File lifetime

Topics: PE reader
Aug 4, 2009 at 10:41 AM

There are a couple of things that bother me about the current treatment of files in CCI.

OpenBinaryDocument in MetadataReaderHost uses MemoryMappedFile to map a file on disk into memory.  The first problem with this is that the subsequent unmapping only occurs on finalisation, which is nondeterministic.  Something, somewhere, should implement IDisposable and let you say "I'm done with this file/assembly, close it down now".

Which brings me to my second issue.  The standard implementation doesn't support reading an assembly that's already in memory (eg. in a MemoryStream).  It's probably a less common case, and it looks like it's doable by overriding OpenBinaryDocument completely, but it'd be nice if there was something standard for it.

If UnmanagedBinaryMemoryBlock supported being given an arbitrary stream instead of a filename, and if it implemented IDisposable, then that'd go a long way.

(This came about because I am trying to read assemblies that are packed into a zip file, so they need to be extracted in memory before they can be worked with.  I tried extracting to a temporary folder, which worked well right up until the point where I wanted to delete the folder at the end, and CCI still had the files open.  I'd prefer not having to write the files to disk at all.)

I think I'll try implementing my own version of OpenBinaryDocument now.  Hopefully that will get past this problem.

 

On a completely unrelated note: any chance one of the two PeToPe projects (sample app and test case) could be renamed?  VSX gets grumpy trying to load a solution containing two projects with the same name.

Coordinator
Aug 5, 2009 at 6:53 AM
Edited Aug 5, 2009 at 2:25 PM

The base class for MetadataReaderHost provided in the metadata helper is a work in progress, for some very slow definition of progress. Overridding OpenBinaryDocument is the way to go. The reader will be happy with whatever implementation you give it.

The reader does not presume to take sole ownership of the IBinaryDocument object given to it. Hence it does not expect this document to implement IDisposable, nor will it call Dispose if the document does implement IDisposable. Managing the lifetime of the document is up to the caller, i.e. you.

I agree that MemoryMappedFile should implement IDisposable, as should UnmanagedBinaryMemoryBlock (there is even a comment to that effect). I'll try and get to that sooner rather than later.

When you say "VSX" do you mean VS2010?

 

Aug 5, 2009 at 11:14 AM

No, by VSX I meant Visual Studio Express.

I ended up creating an InMemoryBinaryDocument (implementing IDisposable, IBinaryDocument, and IBinaryDocumentMemoryBlock, and very strongly resembling UnmanagedBinaryMemoryBlock in implementation).  That seems to work fairly well.

It'd be nice if there were a way to avoid the unsafe code, though.  I realise that it's driven by the MemoryMappedFile (and that there isn't really a good way to take advantage of memory mapping without unsafe pointers), but it seems a bit ugly to force the use of unsafe code when not actually using MemoryMappedFile anyway. :)  It's probably not really a big deal unless someone wants to use CCI from a partial trust environment.  (Which I don't.)

Coordinator
Aug 5, 2009 at 2:25 PM

If someone wants to use CCI from a partial trust environment, they'd have to write their own reader, I'm afraid. In general, though, partial trust scenarios seems a little beyond the scenarios CCI is meant to support.

The use of unsafe pointers is more than just convenience. It makes the reader faster and smaller. This matters quite a bit for a metadata importer used by fast compilers used in a Rapid Application Development environment.

Aug 6, 2009 at 12:18 PM

What is the required lifetime of an IBinaryDocumentMemoryBlock?  Can it be destroyed (and its memory freed) as soon as the PeReader is constructed, or after loading the assembly, or only when no further use of metadata types obtained from that document will occur?

Coordinator
Aug 6, 2009 at 2:21 PM

The last option. The document provides the backing store for much of the reader's object model. Also, the reader constructs the object model on demand.

Coordinator
Sep 24, 2010 at 10:28 PM

The MetadataReaderHost object now implements IDisposable. If you call Dispose on the host object, all files opened by it will be closed and all umanaged memory allocated with it will be released. See Change Set 51451.