Skip to main content
D
Dragon Drop
Participating Frequently
January 24, 2011
Question

How do I use fileStream without reading the file into memory?

  • January 24, 2011
  • 1 reply
  • 4064 views

I am writing a process where users will need to select a file that far exceeds their availble RAM and have that file broken up into small chunks (for upload).

I'm able to create a File reference to said file, but when I try to pass it to the fileStream, it appears to try to read the thing into memory before acting on it.

Is there a way to get fileStream to just take the reference to the file and then utilize readBytes the way it's documented?

Much obliged, in advance.

Here is my code... just in case I'm doing something blatantly stupid...

//this is being called by the File callback, uplon the user browsing over to the selected file.

private function selectHandler(event:Event):void {
            var file:File = File(event.target);
            trace("selectHandler: name=" + file.name );
           
            var stream:FileStream = new FileStream();
           
            var f:File = event.target as File;
            stream.open(f, FileMode.READ); //at this point, the program will lock up if the file exceeds the computer's memory buffer.
            var bytes:ByteArray = new ByteArray();
            stream.readBytes(bytes,0,1024);
            trace(bytes);
            stream.close();
}

-Dr.D

This topic has been closed for replies.

1 reply

chris.campbell
Community Manager
chris.campbellCommunity Manager
Community Manager
January 24, 2011

Hi Dr. D,

Could you try using OpenAsync() instead of Open()?  If I recall, we have a bug on very large (2gb+) files on 32bit systems with Open() that is not present with OpenAsync().  I took a look at the docs and it does appear that reading the file in immediately is as designed:

If the fileMode parameter is set to FileMode.READ or FileMode.UPDATE, AIR reads data into the input buffer as soon as the file is opened, andprogress and open events are dispatched as the data is read to the input buffer.

Chris

D
Dragon DropAuthor
Participating Frequently
January 25, 2011

Hi Chris,

I've tried using openAsync and then reacting to the onProgress event but I'm not certain what I'm seeing.

The two approaches I assumed would work were to open the stream and then alter the position property and do a read. Or to use the reabBytes() method and pass in a distant offset (I'm toying with approx. the 200meg mark in the file, this is 10Gig file).

However, the entire process hangs for a while (about 10 seconds) and then I do get some output, but the whole thing seems very hinky and unstable.

In an ideal world the stream would not actually read any more than I'm asking it to read and only from the position from which I'm asking it to read.  Instead, it's doing stuff behind the scenes that I have no control over or visibility into, so, I'm starting to think that maybe Flash is not the right tool for this job.

D
Dragon DropAuthor
Participating Frequently
January 26, 2011

Ok, I think the key is to set the readAhead property and then call openAsync().  I just tried a sample app (see attached) on a 4gb file and the app did not lock up or run out of memory.  Give it a try when you get a chance and let me know how it goes.

Thanks,

Chris


Well... we're definitely getting closer!

Ok, so the issue I'm seeing now is what appears like inadequate garbage collection.

I'm using readBytes( myByteArray, startPos, sliceSize);

This provokes the continuation of the read (otherwise the read just stops... which is great!)

However, one of two things happen.

Either I don't re-initialize myByteArray in the process, in which case I run out of memory around 640 megs

or, before every read I declare:

myByteArray = new ByteArray()

and this allows me to get to around 1.8 gigs before the system runs out of memory... presumably because the garbage collector isn't keeping up? (I'm guessing obviously).

So the question is... what did you do to vacate that container in your test, or did you use something other than readBytes and if so, what was it?

I'm attaching my code snippet just so that you can tell me I'm doing something wildly stupid (if I am).

        private var total:Number = 0;
        private var b:ByteArray = new ByteArray();
        private function onBytesRead(e:ProgressEvent){
           
            if(e.currentTarget.bytesAvailable >= chunk){
                trace("avail = " + e.currentTarget.bytesAvailable)
                trace ("read " + (total += e.currentTarget.bytesAvailable * 1));
              
                b = new ByteArray();
                e.currentTarget.readBytes(b,total, chunk)
               
               
            }
           
        }   

         //called when user selects a file
         private function selectHandler(event:Event):void {
            var file:File = File(event.target);
           
            stream = new FileStream();           
            var f:File = event.target as File;           
            stream.addEventListener(ProgressEvent.PROGRESS,onBytesRead);
           
            stream.readAhead = chunk;
            stream.openAsync(f, FileMode.READ);           
        }

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices