I process a list of files, and try to upload them to DropBox App using TAdvDropBox.Upload. This code works when I do it one file at a time (e.g. seconds elapse between calls to Upload), but in my production code, I'll get a list of a dozen files or so, and I process them sequentially (e.g. in a loop). Right now, it appears to send up two files, then subsequent calls fail in the TAdvCustomDropBox.UploadInt function, specifically line 1260:
It fails on the first line. Indeed, even when it works, setting a breakpoint at the if statement never gets hit! The parameters being passed into HttpsPut make sense, but it just fails. What can I do to debug this?
Also, are there some logging features, and if so, how do I enable/use them?
Since we use underlying the WinInet API for HTTP connections, it might be due to the maximum concurrent connections to the same domain or it could be due to restrictions from the Dropbox server.
Logging can be enabled by setting the Logging property to True and LogLevel to llDetail. The LOG file is automatically generated in the machine's Documents folder.
How do I do that? Is there a way to tell how many are active? Or when one completes? I certainly would not mind having to wait until the previous transaction is complete!
Good idea, but that doesn't work - the OnUploadProgress event never fires; I've hooked it up but I never hit a breakpoint in it. But you did get me thinking. I am trying to make a service, using SvCom (from www.aldyn-software.com/svcom.html, which is another rather awesome library); is the fact that I'm running in a service interfering somehow with TAdvDropBox? Would hope not, but maybe there is something special I need to do in order to use TAdvDropBox in a Service application?
Hmm. I modified the CloudStorageDemo to support selecting multiple files for uploading, and can easily upload over a dozen files. Oddly, there is a pause between file uploads; I don't know what is causing it, but it seems to allow many uploads. My guess is that is throttling the requests as earlier posts suggested, but I don't know where that is done.
Here is my version that sends many files:
procedure TForm4.UploadBtnClick(Sender: TObject);
var
fn: string;
ci, nci: TCloudItem;
I: Integer;
begin
if OpenDialog1.Execute then
begin
for I := 0 to OpenDialog1.Files.Count - 1 do
begin
fn := opendialog1.Files;
ProgressBar1.Position := 0;
ProgressBar1.Visible := true;
ci := nil;
if Assigned(TreeView1.Selected) then
begin
IsUploading := true;
ci := TCloudItem(TreeView1.Selected.Data);
// when selected item is a file, upload to file parent folder
if ci.ItemType = ciFile then
ci := ci.ParentFolder;
end;
nci := Storage.Upload(ci, fn);
if Assigned(nci) then
CloudTreeViewAdapter1.InitTreeView;
IsUploading := false;
end;
ProgressBar1.Visible := false;
ProgressLabel.Caption := '';
end;
end;
I wonder what is doing the throttling, and how can I do the same in my service app?
I've deleted completely the TTreeView, the TCloudTreeViewAdapter, and the OnUploadProgress method from the CloudStorageDemo app, hoping that might cause it to break like my service app, but the demo still works just fine! So those are clearly not needed to upload multiple files rapidly.
The parameters being passed into HttpsPut make sense, but it just fails. More precisely, in my Service app, this line works the first two times, then fails the third time; in the CloudStorageDemo, this line always works. The time it takes increases with the size of the file, and it appears that HttpsPut does not return until the request is complete; therefore, a tight loop of calls to TAdvCustomDropBox.Upload seems a valid use case - I've demonstrated that with my hacked CloudStorageDemo.
So is it being a Service that is the problem here? What else can I try? Help, please!
Can you watch what the HTTP response is after failure? Can you inspect the resdat value, hopefully this contains some information from Dropbox about the failure
I finally found a clue! After enabling detailed logs, I attempted an upload of four files with the demo, which of course succeeded. What I got in the logfile was four pairs of "PUT"/"PUT RESULT" pairs.
Then, I attempted the same (four files, detailed logging) with my Service app; oddly, this time worked, but the log messages were a bit weird. Instead of the four pairs, I got this:
PUT
PUT RESULT
PUT
PUT
PUT RESULT
PUT
PUT RESULT
In other words, four PUTs, only three PUT RESULTs, and out-of-order messages. I tried again, this time 10 files, and got the following:
PUT
PUT RESULT
PUT
PUT
PUT RESULT
PUT
PUT
Note at this point, five PUTs were seen, but only two PUT RESULTs; up in the cloud, however, the first four files were made it! After the fifth message, there was a long pause (30 seconds, maybe?), then some more PUTs showed up (don't know about the relative timing there).
Given everything I've seen, it appears that the call to HttpsPut() (line 1260 of CloudCustomDropBox.pas, in the TAdvCustomDropBox.UploadInt function) seems to not return reliably; I'm guessing it returns prematurely, and later calls may get the result from a prior call. This may explain why one or two more files make it to the cloud than would be expected from the logs.
DropBox.DoProcessMessages := False;// This is the key!
DropBox.Connect;
It turns out that TCloudBase will call Application.ProcessMessages in the OnUploadProgress and OnDownloadProgress event handlers, which is verboten! Fortunately, those methods only do so if the DoProcessMessages property is True, which is the default; even more fortunately, that property is public! Yahoo!
I suppose I could complain that it isn't documented, but right now, I'm too happy!