Hello,
Is there a limit when loading large local csv files in Miletus?
So far I can not Read / load a large files.
It would be nice to include a function to read a file line by line if a file is to large.
Thanks
Hi,
How large is the file and what are you using to load it?
Line-by-line reading is a good suggestion, we'll put it on our list for investigation.
Hello Tünde,
Large csv file is 138074 KB large. Can not LoadFile
Small csv file is 4359 KB. I can LoadFile into TMSFNCGrid, no issue.
Here is my Miletus code:
If MiletusOpenDialog1.Execute(false)
Then
Begin
If MiletusOpenDialog1.Files.Count > 0
Then
Begin
fn := MiletusOpenDialog1.FileName;
slist := TMiletusStringList.Create;
slist.LoadFromFileAsync(fn,
Procedure
Var
i, j: Integer;
Begin
Try
Stext := slist[0];
// ********Auto detect Delimiter********
Delimiter := ',';
Stext := slist[0];
For j := 1 To 10 Do
Begin
position := LastDelimiter(CSVSeparators[j], Stext);
If position > 0
Then
Begin
Delimiter := Stext[position];
break;
End;
End;
Splitted := Stext.Split([Delimiter]);
TMSFNCGrid1.RowCount := slist.Count;
TMSFNCGrid1.BeginUpdate;
For j := 0 To slist.Count - 1 Do
Begin
Stext := slist[j];
Stext := stringreplace(Stext, '"', '', [rfReplaceAll, rfIgnoreCase]);
Splitted := Stext.Split([Delimiter]);
For i := 0 To Length(Splitted) - 1 Do
Begin
TMSFNCGrid1.cells[i, j] := Splitted[i];
End;
End;
TMSFNCGrid1.EndUpdate;
Finally
Begin
slist.Free;
WebWaitMessage1.hide;
End;
End;
End)
End;
End;
End;
We tested this with a 112MB file and indeed it's too large even for an HTTP request.
We put on our todo list to investigate possible improvements as well as line-by-line reading.
I know with IIS you have specify the maximum size files that can be sent/received. Not sure if that helps.
IIS not involved here.
We are investigating this.
Hello All.
Is there an alternative solution to this issue? What about using nodejs and exporting generated javascript that contains function to read local file line by line. It is doable in nodejs but not sure if I can use it in WebCore?
Regards
Ertan
Hello Tünde.
Is there an alternative solution to this issue? What about using nodejs and exporting generated javascript that contains function to read local file line by line. It is doable in nodejs but not sure if I can use it in WebCore?
Regards
Ertan
Hi,
You probably cannot export and use such functionality given Miletus is not NodeJS based.
With a file this large probably the line by line reading will be the only solution we can offer. Until we can allocate some time on researching this, you could (for testing purposes) try the TWebOpenDialog component which uses the API from the browser directly (HTML input + FileReader) instead of relying on communication between the web and native layers.
See if it's able to open the same large file and if that is the case, what is the performance compared to loading the whole file through the Miletus API.