CCI By Example

Topics: Metadata Model, PDB Reader, PDB Writer, PE reader, PE Writer, Source Model
Nov 1, 2009 at 9:14 PM

Back in a thread from earlier this year (2009) on the project discussion, project coordinator Herman, mentioned wanting to gauge the level of interest in the community to see if you should spend the effort to raise up the CCI projects to released product standards.

I think this is a worthwhile goal (bringing CCI up to SKU level on its own or even absorb into BCL) and the promise of CCI is immense.  (Personally, I'd like to use the CCI Metadata (and evenatually Code and AST) to create an AOP/AOSD system like PostSharp.)

The issue I have with adopting CCI for anything (let alone a whole AOP system) is simply that I don't understand it.  The solution for this, for me at least, is to have very targeted samples that begin extremely simply and progress in complexity, demonstrating all the CCI's capabilities in an instructive way.  Such samples would be as small as possible to illustrate the concept and would be heavily commented.

So something like this:

/// <summary>
/// Demonstrates basic retrieval of types (classes, structs)
/// from an assembly with metadata components
/// </summary>
static void Reflect_all_types_from_an_assembly_read_only()
{
// boilerplate to setup host and load assembly:
var nameTable = new NameTable();
var host = new PeReader.DefaultHost(nameTable);
var coreAssembly = host.LoadAssembly(host.CoreAssemblySymbolicIdentity);

IAssembly assembly
= (IAssembly)host.LoadUnitFrom(@"E:\Projects\...\Prototype000Types.dll");

// this is how you retrieve types from an
// IAssembly
var types = assembly.GetAllTypes();

foreach(INamedTypeDefinition type in types)
{
Console.WriteLine("Retrieved type: {0}", type.Name);
}
}
This, of course, being a very simple example.  So here's how I'd like to contribute: I'd be willing to write these examples - 100's of them if necessary - both as actual code (perhaps in the form of unit/integration tests - but very readable and highly commented as well) so long as I could get support here as to how to actually accomplish the use cases I'd be examining.  I'd also willing to tweet and possibly blog (no promises) about it.  I guess all I'm asking for is a nod that this is a worthwhile idea and that the CCI team would support answering my questions for this effort.   Also, I don't feel strongly about this, but I'd prefer to have the 'by example' code housed in a separate project, if possible (does not need to be coordinated by me).  I know there is a CCI Samples project, but as I read it, that's for full-blown end to end samples, and should probably remain that way. 

So CCI Team:

  • Does the goal of creating a "CCI by example" - a comprehensive set of samples targeted at teaching new devs how to use the CCI sound like a good one?  (Please don't feel the need to be diplomatic if the answer is "no" - but then I'd like to hear how I - a humble average developer - can learn the CCI and share this with others)
  • Would you be willing to set up a project or area in a project I can start doing this (I'm ready now!)
  • If not, would you object to me setting up such a project?
  • Anything else?

Anyway, as for gauging interesting, I hope you will take this as a sign that some of us are very interested in seeing CCI become a living, thriving project!

Thanks,

Richard

Coordinator
Nov 2, 2009 at 2:08 AM

I've never made a secret of the fact that I'd welcome contributions from the CCI community, and moreover that samples and test cases are perhaps the most important things that are still missing, so your post is music to my ears. :-)

I think it makes sense to keep those examples that exist chiefly to illustrate the CCI API close by the code, documentation, discusssion lists and issue list of CCI. In other words, this project. However, we can probably create a solution per sample, and store the solution with the sample, rather than in the root directory.

To get started simply post your (small) samples as patches to the source tree. I'll look them over, give you some feedback and perhaps do a bit of rewriting. Once a sample looks solid, I'll check it in. After we've been through the loop a couple of times, you might want to consider becoming an official developer on the project.

Hopefully you'll also be happy to know that the RiSE team has obtained the services of a Tech Writer for a few months and hopefully this will mean more documentation appearing the in Wiki soon, along with more samples. There is even the prospect of a full blown book on compilers that will make use of CCI/Spec# as the running example.

Nov 2, 2009 at 5:49 PM

That sounds good to start... I am glad to hear you have a writer, I'd love to see the docs fleshed out.   First thing I'll do, tonight, is brainstorm the scenarios/use cases/stories I personally think would be helpful.  Besides the interest in AOP I also am interested in building DSLs so I'll try to think from those two perspectives.

I read the parts about joining the project as a dev... before I start, are you cool with the possibility I may not be able to do that?  I'm not sure of the implications of the agreement you have to sign and if my currently employer would be cool with that.  I know they are fine with outside projects not related to what we do (trading software) just not sure if they paranoid lawyer types would be very cool about it.

Coordinator
Nov 2, 2009 at 8:37 PM

We are aware of the problem posed by the process our own paranoid lawyer types have imposed on becoming a contributor to an MS sponsored CodePlex project. Discussions about it are on-going. If nothing useful comes to pass, I'd be happy to see a non MS sponsored sample project come into being, hopefully as a temporary measure until sanity previals.

Nov 3, 2009 at 6:37 AM

Okay, here's a start at a list of "How To" samples.  This is just brainstorm of the very basics, as far as I can see them, anyone.  I'll concern myself with culling redundant or duplicate items later, but I will work on the basic ones ASAP.  (Remember if some of these seem ridiculously simple, well, I guess they are, but the cost is small and I think that no knowledge should be assumed in these complex metaprogramming matters).  But in any case, you know better probably what you want, so if there is something else you would like to suggest, please let me know. 

One question: I am not an experienced patcher so I'm afraid I'll need more instructions on supplying you with my samples as patches. I have only ever created patches for individual files using WinMerge.  I don't know what you mean when you say: 'provide patches to the source tree'.  As I read that, I'd have to provide a patch for the .sln to add an additional samples project, then subsequent to the first patch, I'd have to provide a patch to the. csproj (if adding a new .cs file) or a patch for any samples .cs files I was updating.  If that is what you want, that's fine.  But I am a patching noob, so I may need help in this area :/

Read these as "How to ....." 

Load an assembly by assembly name
Load an assembly by path to assembly file

Create a new module
Create a new assembly
Add references to an assembly
Add assembly level attributes
Serialize an assembly to disk
Use PEVerify to validate a created assembly

Find types within a module or assembly
Determine if a type is an abstract class
Determine if a type is a sealed class
Determine if a type is an interface
Determine if a type is a delegate
Determine if a type has generic parameters

Create a new type in a new assembly
Create a new type in an existing assembly
Create a new type that derives from a type in a loaded assembly
Create a new type that resolves a generic type parameter of an ancestor
Create a new generic type with an unconstrained type parameter
Create a new generic type with a constrained type parameter

Add an interface to a new type

Find methods on a type
Determine the parameter names and types of a method
Determine if a method parameter passes by address (ref)
Determine if a method has generic parameters
Determine the return type of a method

Find properties on a type
Find events on a type
Find fields on a type
Find constructers on a type (instance and type)

Read the body of a method
Identify the local variables of a method

Create a method on a new type
Create a method on an existing type

Create a method body in a void returning method (nop + ret)
Create a method body in a method that returns a value

Modify the body of an existing method
Insert a call to a method into a method body
Declare a local variable in a method body

... more to come ....

Coordinator
Nov 3, 2009 at 2:35 PM

This seems like a good starting point. Hopefully others will chip in with some ideas as well.

I have no experience with patches either. It seems possible to simply package up a sample project in a Zip file and upload that. I think I'd prefer that.

Nov 21, 2009 at 5:25 PM

Just pinging that I'm still around and have in fact been working on some samples ... I'll try to post something very soon to get feedback pretty soon.

Coordinator
Nov 21, 2009 at 6:05 PM

We've made some progress on the legal niceties for contributing. It is now OK to contribute a file without first doing the assignment agreement, as long as you attach a BSD style license to the contribution. Specifically:

Copyright (c) <YEAR>, <OWNER>
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of the <ORGANIZATION> nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE

 I would prefer to see all contributions have MS-PL attached to them, but we are still waiting for the lawyers to come back to us on that one.

Nov 26, 2009 at 7:39 AM

Cool, well that's some progress, at least!  I hadn't gone through the whole procedure of getting SVN going and getting xUnit and TestDriven working, so at least I've done that now. 

Speaking of, this code model test passes when run via TestDriven.NET inside VS2008, but *fails* in the xUnit test GUI:

[Fact]
public void TestCodeModel()

When you examine the values produced by the xUnit GUI run, the output is indeed different, but I think it's due to ordering issues, perhaps (i.e. Class3 appears first in the output rather than Class2).  Just thought you should know.

Coordinator
Nov 28, 2009 at 5:18 PM

It is indeed an ordering issue and needs to get fixed by making the order deterministic. Its on my list, but not quite on top of the list, I'm afraid.