I've found several examples of using TWebHttpRequest to GET a file, but nothing to do any POST (or other) types of requests.
The manual shows a couple of GET request examples, but no POST requests.
I want to upload a file; it will take a while, so use of a prog-bar will be helpful.
Also, are Headers in the form of 'aaa=bbb' or 'aaa: bbb' ? Headers is a TStringlist, and aaa=bbb is commonly used.
(Please, I'm looking for a code example, not another description. The manual contains a description that seems incomplete to me.)
Another type of request would be fine to use as well, but I'm not sure which ones are supported in WEB Core.
To be able to perform a post, a server needs to be configured to receive the post.
This is different from a get, where every server can just handle the get. We do not have an open server that accepts posts (it would mean anyone can just dump (potentially harmful) content to the server.
If you have a server that is configured to accept a http post, all is needed is:
- set command to post
- set data you want to post in the postdata property
I have a server, and I have a PHP script that's working perfectly with VCL HTTPClient requests. As I said, I don't need more descriptions of how to do it. I cannot copy the code that the HTTPClient uses to work with this component because they are designed very differently, which is why I'm asking for a code example, not further descriptions. The descriptions aren't adding any further information, and the examples I've found for GET requets aren't posting any header info and showing how to attach large files to the request. They send no parameters and get stuff back as JSON data. The descriptions and details are not comparable. I've found six nearly identical examples of a simple GET request, and not a single POST request example.
Surely somebody has used this component to upload a file to a server using a PHP or Python script to receive it? It takes a parameter file=<filename> as one of the POST parameters, along with the file itself.
Here's an example of posting data to a MantisBT (bug tracker) system. As Bruno mentioned, it is just a matter of setting the command to post and adding whatever data you want to send. In this case, the data is just a block of JSON.
WebRequest := TWebHTTPRequest.Create(nil);
WebRequest.URL := 'https://mantis.500foods.com/mantis/pwa_rest.php';
WebRequest.Command := httpPOST;
WebRequest.TimeOut := 5000;
WebRequest.Headers.Clear;
WebRequest.Headers.AddPair('Authorization',AuthToken);
WebRequest.Headers.AddPair('Content-Type','application/json');
WebRequest.PostData := bugreport.ToJSON;
try
BugSubmission := await(TJSXMLHttpRequest, WebRequest.Perform());
BugSubmissionResponse := string(BugSubmission.response);
except on E: Exception do
begin
LogThis(' -- Bug Submission Error: '+E.Message);
end;
end;
In this instance, the "bugreport.ToJSON" contains the various bug-tracking parameters that are needed by the target system, but you would pass whatever data is expected or required by the system you're communicating with. If you have something looking just for a value, it might look something like this.
WebRequest.PostData := 'filename='+SomeFile+'&filetype='+SomeFileType;
The parameters and how they're delineated will be specific to each system.
Thanks, Andrew. How would the file data itself get attached? In another post, Bruno said to load the file as an ArrayOfBytes into a memory buffer. I'm assuming that just putting the filename in the PostData line isn't going to load it directly, especially since we only get a filename from the load file dialog and not a full path name.
The THTTPClient accepts attachments on the request, but this component doesn't seem to support attachments.
Just trying to connect the dots....
TWebOpenDialog also has a Base64 option. Amusingly, one of the HexaGong posts talks about using the same TWebOpenDialog component, but it uses three of its "get" functions for three separate things. Pretty flexible, I'd say!
Personally I'd likely use the Base64 option as it is easy to see and test and print, but that's just me. It could then just be passed as an extra part of the PostData string. But this is dependent on the service.
What is it expecting? If it isn't expecting a Base64 file, then it isn't going to work. If it's expecting a byte array of some kind, then one of the other "get" options for the TWebOpenDialog will hopefully already be available and you can use one of those. But if you're sending the actual file, and it isn't just a text file, you're going to want to use the Base64 or ArrayBuffer option.
Note that attachments might very well be encoded with Base64 - not an obscure thing at all.
MP3 files -- 5-50 MB long. Not anything you'd print or even look at.
Later on, some DOCX and PDF files.
By "see and test and print" I was referring to the idea that it is a string - you don't need to see 5 MB of text to know that it has a value. It can be shown in the browser console (with just a snippet showing and a button to see more). And it can be passed around without having to give any thought as to whether it is passed by value or reference. This makes debugging a great deal easier than if you have a blob or some binary format that isn't particularly viewable in its internal state, and that disappears if you do something with the object it was created from.
I get all of that, I'm just trying to understand how to "attach" a huge "string" of data in an ArrayBuffer or something else created by the WebOpenFile dialog to this thing.
For THTTPClient, I just do this:
FormData.AddField('f', 'type1');
FormData.AddFile('file', tmp_zip_file, 'application/octet-stream');
But tmp_zip_file is a full pathname for the file and AddFile knows to open it, encode it, and attach it in whatever format it chooses. Here those are done partly by the WebOpenFile logic and partly by ...what? The PostData method? Is there only ONE call to PostData allowed? Or one per header line? This looks like it may be a GET request. Can you attach 5-10 MB of data to the URL?
It's clearly an in-memory buffer that's being passed-by-value.
Bruno said it's NOT a POST request, and that the browser is handling it. 5-10 MB is a HUGE payload. What if it's a big .mov or .mp4 video file that's 600 MB long?
Welcome to the carefree lifestyle of JavaScript development. And its not really an optional outlook. You honestly don't care much of the time how big things are. Browsers will be browsers and will handle things... until they can't.
To use TWebOpenDialog, the general idea is that you drop the non-visual component on the form and assign the "Accept" property to something close to what you're interested in the user selecting.
- "image/*" for all browser-supported images formats
- "audio/*" for all browser-supported audio formats
- "video/*" for all browser-supported video formats
- ".jpg, ,jpeg, .png" if you just want JPEG or PNG files
- Whatever other extensions you might be interested in
You can also select whether you'll allow multiple files or just a single file. This is largely a wrapper for the <input type="file"> mechanism in HTML, which has other options as well, mostly related to the camera (which camera, only use camera, etc.). From a button, you can invoke it perhaps by doing something like this.
procedure TForm1.btnLoadFiles(Sender: TObject);
var
i: Integer;
begin
// Open file dialog
await(string, WebOpenDialog1.Perform);
// If files were selected, iterate through them
i := 0;
while (i < WebOpenDialog1.Files.Count) do
begin
WebOpenDialog1.Files.Items[i].GetFileAsBase64;
i := i + 1;
end;
end;
Then, by adding a GetFileAsBase64 event handler to WebOpenDialog1, you do whatever you want to do with the files.
procedure TForm1.WebOpenDialog1GetFileAsBase64(Sender: TObject; AFileIndex: Integer; ABase64: string);
var
WebRequest: TWebHttpRequest;
Submission: TJSXMLHttpRequest;
SubmissionResponse: String;
begin
WebRequest := TWebHTTPRequest.Create(nil);
WebRequest.URL := 'https://your.server.com/some_script.php';
WebRequest.Command := httpPOST;
WebRequest.TimeOut := 5000;
WebRequest.Headers.Clear;
WebRequest.Headers.AddPair('Authorization',AuthToken);
WebRequest.Headers.AddPair('Content-Type','application/json');
WebRequest.PostData := 'filename='+
WebOpenDialog1.Files[AFileIndex].name+
'&file='+
ABase64;
try
Submission := await(TJSXMLHttpRequest, WebRequest.Perform());
SubmissionResponse := string(Submission.response);
except on E: Exception do
begin
console.log('Submission Error: '+E.Message);
end;
end;
end;
1 Like
It is entirely up to the server to set what it is willing to accept. In my first example, MantisBT was the target, which expects JSON (also a string) with a variety of elements that it needs to create a bug report in its system.
Using the convention for URL parameters is another possibility, but only if the script is set up that way.
If you're connecting with an XData server, the parameters are passed as JSON if the endpoint is configured to use POST, and passed as URL parameters if configured to use GET.
And it is probably worth mentioning that this is another distinction. Whether you use POST or GET will depend on what the server is configured to accept. Generally, if you are passing a lot of data (say, more thank 2 KB) you'll need to use POST as historically some browers didn't support URLs longer than that. Probably not the case anymore, but there is still a limit of some kind, typically in the range of KB but not MB. There are also issues related to logging and security - some networks log all incoming URLs, so having a lot of data there may be undesirable from both a performance and security perspective. In other words, if you have more than a handful of primitive parameters, use POST.
Ultimately, you need to know what the server is willing to accept. This is one of the things that make XData and Swagger so awesome - you can see a generated interface that tells you exactly what is needed - and even test it on the spot. Simplifies a lot of this stuff.
For example, it might be that your service requires three parameters. It might even be that the order is specific, with maybe a filename, a filetype of some kind, and then the file as base64. It might not know how to deal with having anything after the file data. Who knows. Depends on the server.
Here's a simplified look at what I'm doing now with VCL code and a php script on my server. I'd like to use this same approach (and script if possible) for this. Its only purpose is to handle uploads of a few different file types and stash them away for later use.
BTW, this is running on a Linux host.
// on the Delphi side, using THTTPClient:
FormData.AddField('f', 'type1');
FormData.AddFile('zipfile', tmp_zip_file, 'application/octet-stream');
----------------------------------------
// The php script on the server side that's receiving this data
// does so as follows:
if (isset($_GET['versID'])) {
$versionID = 'V1.2.3.4';
http_response_code(200);
echo $versionID;
exit;
}
if (!isset($_POST['f'])) {
http_response_code(400);
die('File ID missing.');
}
$fileID = $_POST['f'];
// fileID might identify the file as a zip file,
// or an audio file (what I'm trying to add now)
if (!isset($_FILES['zipfile'])) {
http_response_code(400);
die('Zip file missing.');
}
$uploadedFile = $_FILES['zipfile'];
$sourceFile = $uploadedFile['tmp_name'];
$toFile = $destinationFolder . $uploadedFile['name'];
if (!move_uploaded_file($sourceFile, $toFile)) {
http_response_code(500);
die("Failed to move the file $sourceFile into the destination folder: $toFile\n");
}
// do some other stuff with the zip file...
http_response_code(200);
echo 'File uploaded and processed successfully.';
This means this PHP script accepts multipart form-data.
At this moment, TWebHttpRequest has no built-in interface to add data as multipart form-data.
It can be done in the browser following the top answer here:
Either integrate this as JavaScript code in an ASM block or use the Pascal wrappers in web.pas , i.e. TJSFormData & TJSXMLHttpRequest
1 Like
Well, I can make a separate script for this if need be. I don't know what the correct way of phrasing this is, but what does the PHP code want to look for to accept whatever TWebHttpRequest is sending it? I know they both have to agree, but all you've said is that the browser is handling it, and I don't know what "packaging" is being used by the browser so I can tell the PHP code how to unwrap it.
There are two sides to this equation, and usually there's something in the client side that sets up what the server side needs to use to consume the data. I personally don't care what it is. I just don't know what IT is when all I have to go on is, "the browser handles it" -- but not without some cooperation on the server side!
A couple of years ago, I learned I'm somewhat autistic (Asperger's Syndrome) and my brain does not handle stuff like this very well. I like things neat and orderly, and vague or incomplete descriptions send me down a rabbit-hole that can end up in exhaustive depth-first searches for things that some people would figure out really quickly. I'm not stupid, it's just how my brain seems to work, and I try my best to avoid it, which is why I ask to see coding examples -- they're clear, unambiguous, and don't send me down deep networks of rabbit-holes in search of something that's already not obvious to me (but with any luck, I'll know it when I see it).
Using the Base64 example, you could write your PHP script to just check for another regular parameter. You're checking for f as a parameter using $_POST['f'], so you could check for something like b64file using $_POST['b64file']. You could then convert the base64 file in PHP.
$data = base64_decode($b64file);
$file = fopen($f, 'w');
fwrite($file, $data);
fclose($file);
ok, that's easy to do. Thanks!
If you get it working, maybe post the code you've used for both sides to finish up this thread.
1 Like