During the development of Webiny we have tried out and changed several ways of how files are uploaded to the server. Each time we faced a different problem, from usability and security to memory usage problems.
The first attempt was to give a user several file input fields where he can select files one by one. Here we faced a problem where he needed to click several times for each file he wanted to upload, which was a very bad approach.
Our second attempt was using a flash based file upload plugin called SWFUpload which proved a quite good solution, if you had flash player installed. Here we have had several issues, and one was problematic in particular. Namely flash doesn't inherit current session, instead it creates a new session, which caused the loss of current users login cookie. This problem could be avoided by passing the current session id and login cookie through POST parameters with each file upload, but that could also be a potential security issue. Therefore we decided that SWFUpload isn't also a good solution. So there was nothing else left but to develop our own file upload mechanism.
The Technology
HTML5 has been around for quite a while now. Several systems already use it, but there are many more that don't, mostly because it's not compatible with older browsers (read Internet Explorer). When it comes to Webiny, we have a rule for system administration that all code must be compatible with the latest version of browsers, so no backwards compatibility, in the means of HTML, CSS and JS. This rule had to be applied so we could use new cool stuff that came with HTML5 & CSS3.
The Code
We have developed a JavaScript object that attaches certain observers to a defined file-input field. Those observers then trigger some actions and call callback functions that control the file upload process. But in this blog post, we'll only concentrate on how to upload files using XMLHttpRequest (v2) and HTML5.
HTML
The HTML code is essential, just a normal input field, with an optional "multiple" attribute. The "multiple" attribute was introduced in HTML5 and allows the user to select multiple files by holding SHIFT or CTRL key while selecting files on his computer.
JavaScript
The upload function uses FileReader class, which is included in HTML5 API and supported on all modern browsers. The FileReader is responsible for reading the file contents. Once the file is read, we start the upload process using XHR request. Since we stream the file contents to the server, we can track the upload progress and read the file contents from the input stream on the server side. In order for all that to be possible, we have to send the POST parameters in an binary format. Firefox browser supports sendAsBinary function on XHR objects, while other browsers don't, so we had to create a function for streaming binary data that would work on all browsers, thus the 'mySendAsBinary' function in the code below. While file is uploading, browser triggers a callback function called 'progress' on xhr.upload object. Inside that function you have the total size of file that is currently uploading, and the size that is already sent to the server. With those two numbers you can get the upload percentage.
function upload(fileInputId, fileIndex)
{
// take the file from the input
var file = document.getElementById(fileInputId).files[fileIndex];
var reader = new FileReader();
reader.readAsBinaryString(file); // alternatively you can use readAsDataURL
reader.onloadend = function(evt)
{
// create XHR instance
xhr = new XMLHttpRequest();
// send the file through POST
xhr.open("POST", 'upload.php', true);
// make sure we have the sendAsBinary method on all browsers
XMLHttpRequest.prototype.mySendAsBinary = function(text){
var data = new ArrayBuffer(text.length);
var ui8a = new Uint8Array(data, 0);
for (var i = 0; i < text.length; i++) ui8a[i] = (text.charCodeAt(i) & 0xff);
if(typeof window.Blob == "function")
{
var blob = new Blob([data]);
}else{
var bb = new (window.MozBlobBuilder || window.WebKitBlobBuilder || window.BlobBuilder)();
bb.append(data);
var blob = bb.getBlob();
}
this.send(blob);
}
// let's track upload progress
var eventSource = xhr.upload || xhr;
eventSource.addEventListener("progress", function(e) {
// get percentage of how much of the current file has been sent
var position = e.position || e.loaded;
var total = e.totalSize || e.total;
var percentage = Math.round((position/total)*100);
// here you should write your own code how you wish to proces this
});
// state change observer - we need to know when and if the file was successfully uploaded
xhr.onreadystatechange = function()
{
if(xhr.readyState == 4)
{
if(xhr.status == 200)
{
// process success
}else{
// process error
}
}
};
// start sending
xhr.mySendAsBinary(evt.target.result);
};
}
This is only the essential code needed for you to understand how this works. You can easily expand it with the functionality to add the validation of file types, file size, additional callbacks like when the upload has started, when a new file added to the queue and similar.
In Webiny file upload we have integrated the following callbacks:
- onFileAdded - called when a file is added to the upload queue
- onQueueInsertDone - called when all files are added to the queue
- onClearQueue - called after all files have been cleared from the queue
- onFileDiscarded - called when a file has been discarded from the queue because of invalid extension or its size
- onStartUpload - called when the startUpload() has been issued
- onFileUploadStart - called when a specific file has started uploading
- onUploadProgress - called several time while a specific file is uploading
- onFileUploaded - called when a specific file was uploaded
- onQueueDone - called when all files from the queue have been uploaded
Hope this gives you additional ideas of what can be achieved using this method for uploading files. Here is a screenshot of how the file uploader looks on Webiny.
Server Side (PHP)
The code on the server side is quite simple, just read the data from the input stream and save it somewhere on the server.
// read contents from the input stream
$inputHandler = fopen('php://input', "r");
// create a temp file where to save data from the input stream
$fileHandler = fopen('/tmp/myfile.tmp', "w+");
// save data from the input stream
while(true) {
$buffer = fgets($inputHandler, 4096);
if (strlen($buffer) == 0) {
fclose($inputHandler);
fclose($fileHandler);
return true;
}
fwrite($fileHandler, $buffer);
}
// done
You can additionally improve this function by passing desired file-name, under which it should be saved, from JavaScript. Also when the upload is done, a JSON response is suitable so you can easily show the results on client side.
Why streams and not just file_put_contents?
Using PHP input:// stream the memory footprint on the server is <1MB, on an upload file size of 12MB, compared to normal file upload (without binary streams) where memory footprint reaches over 60MB on the same file size. If you don't want to use streams, but you are concerned about the memory usage, one good alternative is to use an event based server like nginx or lighttpd which would outperform apache when it comes to file upload.
Do you need a web site?
Webiny makes it easy to create and manage websites.
You'll get a ton of additional features.
Best of all, it's free!
Author: Sven Al Hamad
Founder, Chief Technology Officer
Software engineer, project manager, team leader and PHP expert.
"I'm a big perfectionist when it comes to user experience and work flow. I go with the thought that one click is twice as better than two clicks."
Edi Budimilic said 1 year ago:
This is great, I like the input:// stream. It's awesome!
Sven said 1 year ago:
Thanks Edi,
keep following us, we have much more knowledge to share.
Daniel André said 1 year ago:
Thanks for the easy explanation of html5 file uploads. Just a tip about the temp file to keep the example useful across platform - it would probably be better to use the php tmpfile() function, so you know you're getting a random temporary file each time, and don't overwrite other files being uploaded at the same time.
Once again, thanks for the easy-to-follow article.
Sven said 1 year ago:
Hi Daniel,
Thank for the comment, and a good tip ;)
Mike said 10 months ago:
This is great. I was wondering if you could hint at how you could send a response back from php to the upload() function saying when each file was uploaded completely. something like echo json_encode(array('message' => "$filename was uploaded")) for each file that was uploaded, because it will be multiple files.
how would you do that in the php? and you mentioned onFileUploaded as a callback. thanks a million.
Sven said 10 months ago:
Hi Mike,
It's very simple to send back the response, at the end of the upload function in PHP, just do something like:
die(json_encode(array('msg'=>'Upload done', 'error'=>false)));
And inside your javascript function, at the part here I wrote '// process success' you can get that JSON data, and here is how:
try{
// here you have your JSON response
var result = xhr.responseText.evalJSON();
// alert the message
alert(result.msg);
}catch(err)
{
// if you enter here, something is wrong
}
Hope it helps.
coder said 10 months ago:
Hi, im searching all over the net trying to find a way to upload large files, 500MB and above. I came across this and decided to give it a try. I successfully upload small files, but when i tried uploading a 700MB video file, the browser just crashed, and it wont allow me to upload that large file. Im using PHP.
If any one could point me in the right direction it would be much appreciated.
coder said 10 months ago:
ohhhh i am using chrome for my testing.
Sven said 10 months ago:
Hi Coder,
This solution won't probably work for you, mostly because when you select a file, before sending it to the server, the file is read and converted into a binary string. Now if you select such a large file, your browser will probably crash in that process.
In that case I suggest that you try a flash-based uploader like uploadify in a combination with a event-driven web server such as nginx to keep the memory usage low as possible.
coder said 10 months ago:
Thanks much Sven. In my searching I've heard of nginx been mentioned before, so it seems like that might be my solution.
Thanks, will look into those.
coder said 10 months ago:
Sven,
Can you recommend a good hosting provider that will provide me with those services (php 5.x, mysql 5.5.x, nginx)
Sven said 9 months ago:
I really never rented a specialized nginx server. I would suggest using some VPS with ubuntu or centos and then just installing nginx in front of apache. This way he is the one in charge of handing all the requests. In this setup you still have the possibility to route necessary requests to apache.
There are plenty of tutorials out there, just google them :)
coder said 9 months ago:
Yes will do. Google as always been my friend :-)
Carlos said 8 months ago:
Excelent article.
Thanks a lot!.
Adrian said 6 months ago:
Hello,
I have newbie question about using your upload function. From where does this function obtain the parameters called fileInputId and fileIndex?
This is the form what I tried to use for uploading, but unfortunately it doesn't work, because the upload function doesn't get any parameters:
Adrian said 6 months ago:
There is a slight mistake in closing the input stream: fclose(inputHandler);
But anyway thanks for sharing this with us, it is a very smart solution.
Sven said 6 months ago:
Hi Adrian,
Thank you for highlighting the mistake and enabling me to correct it.
In reply to your previous comment, did you manage to get your upload function working?
Adrian said 6 months ago:
Hi Sven,
Yes, I had to dig in html5 to understand it, but its working now properly. Still I have a question: is there any way to send variables from the javascript function to the php script other than the get method?
I mean I would like to pass information to the php script like the name and the extension of the uploaded file.
Viktor said 5 months ago:
Sven, thanks for sharing. This method will make life much easier for me!
Maj said 4 months ago:
Can you please provide us more information with regards to how secure your code is to upload files? Thank you in advance.
Fulvio said 4 months ago:
I have an error (on console), to line 20 of first snippet: "Uncaught TypeError: undefined is not a function" o.O'
*XMLHttpRequest.mySendAsBinary
*reader.onloadend
Thanks' for article
Sven said 4 months ago:
@Maj:
The code is mostly JavaScript based, so you should not have any issues with security.
When it comes to PHP side, you can always read the headers of the uploaded file to determine its type, and accordingly handle the file.
@Fulvio
This might be caused with the latest Firefox update (v18) where they removed MozBlobBulder form their library. I will update the blog post shortly with a fix for that.
goni said 3 months ago:
U should update this code to new constructor Blob(). This is not working anymore in chrome.
Sven said 3 months ago:
The code is updated and now it supports the Blob() constructor.