html5-11-blogposts-727.jpg

Webiny file upload with HTML5 and AJAX using PHP streams

Webiny file upload with HTML5 and AJAX using PHP streams

34 comments

This blog post gives an overview how to do asynchronous file upload using new possibilities given in HTML5. Besides that, we talk about how to use PHP I/O streams to keep the memory footprint low as possible during upload of large files. This principle is also used for uploading files in Webiny administration.

During the development of Webiny we have tried out and changed several ways of how files are uploaded to the server. Each time we faced a different problem, from usability and security to memory usage problems.
The first attempt was to give a user several file input fields where he can select files one by one. Here we faced a problem where he needed to click several times for each file he wanted to upload, which was a very bad approach. 
Our second attempt was using a flash based file upload plugin called SWFUpload which proved a quite good solution, if you had flash player installed. Here we have had several issues, and one was problematic in particular. Namely flash doesn't inherit current session, instead it creates a new session, which caused the loss of current users login cookie. This problem could be avoided by passing the current session id and login cookie through POST parameters with each file upload, but that could also be a potential security issue. Therefore we decided that SWFUpload isn't also a good solution. So there was nothing else left but to develop our own file upload mechanism.

The Technology

HTML5 has been around for quite a while now. Several systems already use it, but there are many more that don't, mostly because it's not compatible with older browsers (read Internet Explorer). When it comes to Webiny, we have a rule for system administration that all code must be compatible with the latest version of browsers, so no backwards compatibility, in the means of HTML, CSS and JS. This rule had to be applied so we could use new cool stuff that came with HTML5 & CSS3. 

The Code

We have developed a JavaScript object that attaches certain observers to a defined file-input field. Those observers then trigger some actions and call callback functions that control the file upload process. But in this blog post, we'll only concentrate on how to upload files using XMLHttpRequest (v2) and HTML5.  

HTML

The HTML code is essential, just a normal input field, with an optional "multiple" attribute. The "multiple" attribute was introduced in HTML5 and allows the user to select multiple files by holding SHIFT or CTRL key while selecting files on his computer.


JavaScript

The upload function uses FileReader class, which is included in HTML5 API and supported on all modern browsers. The FileReader is responsible for reading the file contents. Once the file is read, we start the upload process using XHR request. Since we stream the file contents to the server, we can track the upload progress and read the file contents from the input stream on the server side. In order for all that to be possible, we have to send the POST parameters in an binary format. Firefox browser supports sendAsBinary function on XHR objects, while other browsers don't, so we had to create a function for streaming binary data that would work on all browsers, thus the 'mySendAsBinary' function in the code below. While file is uploading, browser triggers a callback function called 'progress' on xhr.upload object. Inside that function you have the total size of file that is currently uploading, and the size that is already sent to the server. With those two numbers you can get the upload percentage. 

function upload(fileInputId, fileIndex)
		{
			// take the file from the input
			var file = document.getElementById(fileInputId).files[fileIndex]; 
			var reader = new FileReader();
			reader.readAsBinaryString(file); // alternatively you can use readAsDataURL
			reader.onloadend  = function(evt)
			{
					// create XHR instance
					xhr = new XMLHttpRequest();
					
					// send the file through POST
					xhr.open("POST", 'upload.php', true);

					// make sure we have the sendAsBinary method on all browsers
					XMLHttpRequest.prototype.mySendAsBinary = function(text){
						var data = new ArrayBuffer(text.length);
						var ui8a = new Uint8Array(data, 0);
						for (var i = 0; i < text.length; i++) ui8a[i] = (text.charCodeAt(i) & 0xff);
			
                        if(typeof window.Blob == "function")
			            {
			                 var blob = new Blob([data]);
			            }else{
			                 var bb = new (window.MozBlobBuilder || window.WebKitBlobBuilder || window.BlobBuilder)();
			                 bb.append(data);
			                 var blob = bb.getBlob();
			            }

						this.send(blob);
					}
					
					// let's track upload progress
					var eventSource = xhr.upload || xhr;
					eventSource.addEventListener("progress", function(e) {
						// get percentage of how much of the current file has been sent
						var position = e.position || e.loaded;
						var total = e.totalSize || e.total;
						var percentage = Math.round((position/total)*100);
						
						// here you should write your own code how you wish to proces this
					});
					
					// state change observer - we need to know when and if the file was successfully uploaded
					xhr.onreadystatechange = function()
					{
						if(xhr.readyState == 4)
						{
							if(xhr.status == 200)
							{
								// process success
							}else{
								// process error
							}
						}
					};
					
					// start sending
					xhr.mySendAsBinary(evt.target.result);
			};
		}

This is only the essential code needed for you to understand how this works. You can easily expand it with the functionality to add the validation of file types, file size, additional callbacks like when the upload has started, when a new file added to the queue and similar.

In Webiny file upload we have integrated the following callbacks:

  • onFileAdded - called when a file is added to the upload queue
  • onQueueInsertDone - called when all files are added to the queue
  • onClearQueue - called after all files have been cleared from the queue
  • onFileDiscarded - called when a file has been discarded from the queue because of invalid extension or its size
  • onStartUpload - called when the startUpload() has been issued
  • onFileUploadStart - called when a specific file has started uploading
  • onUploadProgress - called several time while a specific file is uploading
  • onFileUploaded - called when a specific file was uploaded
  • onQueueDone - called when all files from the queue have been uploaded 

 

Hope this gives you additional ideas of what can be achieved using this method for uploading files. Here is a screenshot of how the file uploader looks on Webiny.

Server Side (PHP)

The code on the server side is quite simple, just read the data from the input stream and save it somewhere on the server.

// read contents from the input stream
$inputHandler = fopen('php://input', "r");
// create a temp file where to save data from the input stream
$fileHandler = fopen('/tmp/myfile.tmp', "w+");

// save data from the input stream
while(true) {
	$buffer = fgets($inputHandler, 4096);
	if (strlen($buffer) == 0) {
		fclose($inputHandler);
		fclose($fileHandler);
		return true;
	}

	fwrite($fileHandler, $buffer);
}

// done

You can additionally improve this function by passing desired file-name, under which it should be saved, from JavaScript. Also when the upload is done, a JSON response is suitable so you can easily show the results on client side.

Why streams and not just file_put_contents?

Using PHP input:// stream the memory footprint on the server is <1MB, on an upload file size of 12MB, compared to normal file upload (without binary streams) where memory footprint reaches over 60MB on the same file size. If you don't want to use streams, but you are concerned about the memory usage, one good alternative is to use an event based server like nginx or lighttpd which would outperform apache when it comes to file upload. 

34 Comments

  • Edi Budimilic
    2 years ago
  • This is great, I like the input:// stream. It's awesome!

  • Sven
    2 years ago
  • Thanks Edi,

    keep following us, we have much more knowledge to share.

  • Daniel André
    2 years ago
  • Thanks for the easy explanation of html5 file uploads. Just a tip about the temp file to keep the example useful across platform - it would probably be better to use the php tmpfile() function, so you know you're getting a random temporary file each time, and don't overwrite other files being uploaded at the same time.

    Once again, thanks for the easy-to-follow article.

  • Sven
    2 years ago
  • Hi Daniel,

    Thank for the comment, and a good tip ;)

  • Mike
    2 years ago
  • This is great. I was wondering if you could hint at how you could send a response back from php to the upload() function saying when each file was uploaded completely. something like echo json_encode(array('message' => "$filename was uploaded")) for each file that was uploaded, because it will be multiple files.

    how would you do that in the php? and you mentioned onFileUploaded as a callback. thanks a million.

  • Sven
    2 years ago
  • Hi Mike,

    It's very simple to send back the response, at the end of the upload function in PHP, just do something like:

    die(json_encode(array('msg'=>'Upload done', 'error'=>false)));

    And inside your javascript function, at the part here I wrote '// process success' you can get that JSON data, and here is how:

    try{
    // here you have your JSON response
    var result = xhr.responseText.evalJSON();

    // alert the message
    alert(result.msg);
    }catch(err)
    {
    // if you enter here, something is wrong
    }

    Hope it helps.

  • coder
    2 years ago
  • Hi, im searching all over the net trying to find a way to upload large files, 500MB and above. I came across this and decided to give it a try. I successfully upload small files, but when i tried uploading a 700MB video file, the browser just crashed, and it wont allow me to upload that large file. Im using PHP.

    If any one could point me in the right direction it would be much appreciated.

  • coder
    2 years ago
  • ohhhh i am using chrome for my testing.

  • Sven
    2 years ago
  • Hi Coder,

    This solution won't probably work for you, mostly because when you select a file, before sending it to the server, the file is read and converted into a binary string. Now if you select such a large file, your browser will probably crash in that process.

    In that case I suggest that you try a flash-based uploader like uploadify in a combination with a event-driven web server such as nginx to keep the memory usage low as possible.

  • coder
    2 years ago
  • Thanks much Sven. In my searching I've heard of nginx been mentioned before, so it seems like that might be my solution.

    Thanks, will look into those.

  • coder
    2 years ago
  • Sven,

    Can you recommend a good hosting provider that will provide me with those services (php 5.x, mysql 5.5.x, nginx)

  • Sven
    2 years ago
  • I really never rented a specialized nginx server. I would suggest using some VPS with ubuntu or centos and then just installing nginx in front of apache. This way he is the one in charge of handing all the requests. In this setup you still have the possibility to route necessary requests to apache.

    There are plenty of tutorials out there, just google them :)

  • coder
    2 years ago
  • Yes will do. Google as always been my friend :-)

  • Carlos
    1 year ago
  • Excelent article.

    Thanks a lot!.

  • Adrian
    1 year ago
  • Hello,
    I have newbie question about using your upload function. From where does this function obtain the parameters called fileInputId and fileIndex?

    This is the form what I tried to use for uploading, but unfortunately it doesn't work, because the upload function doesn't get any parameters:







  • Adrian
    1 year ago
  • There is a slight mistake in closing the input stream: fclose(inputHandler);
    But anyway thanks for sharing this with us, it is a very smart solution.

  • Sven
    1 year ago
  • Hi Adrian,

    Thank you for highlighting the mistake and enabling me to correct it.

    In reply to your previous comment, did you manage to get your upload function working?

  • Adrian
    1 year ago
  • Hi Sven,

    Yes, I had to dig in html5 to understand it, but its working now properly. Still I have a question: is there any way to send variables from the javascript function to the php script other than the get method?
    I mean I would like to pass information to the php script like the name and the extension of the uploaded file.

  • Viktor
    1 year ago
  • Sven, thanks for sharing. This method will make life much easier for me!

  • Maj
    1 year ago
  • Can you please provide us more information with regards to how secure your code is to upload files? Thank you in advance.

  • Fulvio
    1 year ago
  • I have an error (on console), to line 20 of first snippet: "Uncaught TypeError: undefined is not a function" o.O'
    *XMLHttpRequest.mySendAsBinary
    *reader.onloadend

    Thanks' for article

  • Sven
    1 year ago
  • @Maj:
    The code is mostly JavaScript based, so you should not have any issues with security.
    When it comes to PHP side, you can always read the headers of the uploaded file to determine its type, and accordingly handle the file.

    @Fulvio
    This might be caused with the latest Firefox update (v18) where they removed MozBlobBulder form their library. I will update the blog post shortly with a fix for that.

  • goni
    1 year ago
  • U should update this code to new constructor Blob(). This is not working anymore in chrome.

  • Sven
    1 year ago
  • The code is updated and now it supports the Blob() constructor.

  • Ehsan
    1 year ago
  • Thanks, it seems great, but for me as an novice can you give a complete working sample to download.

  • Derrick
    11 months ago
  • You mentioned above: "You can additionally improve this function by passing desired file-name, under which it should be saved, from JavaScript."

    How might that look on the JS side and then how would PHP parse that data given the streaming method above?

  • EnA
    9 months ago
  • Thank you for this beautiful work
    For aspx server side code



    Imports System
    Imports System.Web

    Public Class upload : Implements IHttpHandler

    Public Sub ProcessRequest(ByVal context As HttpContext) Implements IHttpHandler.ProcessRequest
    'context.Response.ContentType = "text/plain"

    Dim ad As String = context.Request("ad").ToString()
    Dim B() As Byte
    Dim Z As Long = 0
    Z = context.Request.ContentLength
    B = context.Request.BinaryRead(Z)
    System.IO.File.WriteAllBytes("C:TMP"

  • EnA
    9 months ago
  • Another site on the subject

    http://www.html5rocks.com/en/tutorials/file/dndfiles/

  • Joey
    8 months ago
  • YOU ARE MY HERO!!!! WORKS AWESOME!!! BLESSINGS

  • Hao
    6 months ago
  • Hi,

    VPS disabled fopen, so I used http://pear.php.net/package/PHP_Compat, so I can use file_get_content at least. The uploading is fine, but the post has "301 forbidden" error. Any idea?

  • Alex
    6 months ago
  • Use the XHR is like use AJAX calls in jQuery? So, for every image you want to upload you make a call?

  • Shawn Welch
    1 month ago
  • How do you get the name of the uploaded filejQuery203016803913446707486_1401808458389?

    Also, the copy and paste from this site is annoying as it appends a link.

  • Dan
    1 month ago
  • Great informative tutorial, well done :)

Leave a comment

*

*

*