Introduction
The article is not a rant about the current state of tools like Visual Studio 2008 Designers, code generators like SqlMetal or SqlTac or home grown code generators.
Instead it’s an article on the type of code generation tools that developers in the field need in order to deliver maintainable, documented, multi-tier applications that are UI and data source agnostic. This article also provides good “real world’ methods for solving the problems presented here.
Code generation has always been (at least in my 19 years in IT) an imperative. Starting with metadata, the generator should be able to crank out documented database objects, documented business entities, documented business layer objects for processing the business entities and documented data layer code.
Documentation
You’ve noticed my repeated use of the word, documented. This means that the generated classes have good intellisense for each class, class member and class method and that this same documentation can be used to generate a help file.
The documentation requirement is front and center because it has been largely overlooked by code generators. One reason for this is because there is no industry standard for storing metadata.
Some store it in extended properties, others in separate tables, others in Word Documents, don’t laugh I’ve seen this.
Rather than getting myself tossed into that FIRE, I would propose that Microsoft publish a Typed Dataset .xsd file that they and other vendors or developers can use to place their metadata in. Then when the generation tools run, they can extract the required information from the Typed Dataset and include it in the generated code output. This would include the Visual Studio Designers, SqlMetal, SqlTac, etc.
The standard Typed Dataset solves the problem of disparate metadata. Developers would then just generate this metadata Type Dataset from their current datasource(s) and persist it to disk. The generation tool would then read this file during the generation process. Having the generation tools all read metadata from a standard Dataset at generation time solves the problem and allows all developers to keep their metadata where they currently have it. This decoupling of the metadata store also makes it very easy for code generator authors to focus on writing their code generator and not having to worry about integrating user metadata into their products. This also makes it very easy for developers authoring templates as they can now refer to their metadata inside their templates using a type safe name. Something about a win-win situation comes to mind…
This Typed Dataset structure should be simple. Almost any metadata store would currently have data or could quickly be modified to populate the following Typed Dataset:
- Class/Table Name
- Namespace – used to place the class in a certain Namespace (a serious oversight in SqlMetal)
- Base Class – used to assign a base class for the generated class
- Select Returns – name of class to return from the Select stored procedure.(see Generated Data Access Code below)
- Select All Returns – name of class to return from the SelectAll stored procedure.(see Generated Data Access Code below)
- Summary
- Remarks
- Properties
- Property/Column Name
- Summary
- Remarks
- Is Pulldown Key (see Pulldown Below)
- Pulldown Field Position (see Pulldown Below)
- Methods (optional)
- Method Name
- Summary
- Remarks
You will noticed that I did not include the Parameters, Exceptions or Returns fields in the above Typed Dataset. That is because the code generator should generate the documentation for the parameters, exceptions and the return type.
If the Method metadata was not supplied, then the generator will generate the summary section of the comments.
Not all classes belong to the same namespace. The ability to have the generator place class entity and class entity business manipulation classes in assigned namespaces is a non-negotiable requirement.
This is where the water can get muddy. Let me explain.
We have an executable or web site that handles accounting, utility billing, tax and cashiering. To me, this looks like at least 4 different projects with a UI project. Each of the business layers is in its own namespace and project.
Now we generate our code using SqlMetal and we get one DataContext object with lots of entity objects that are all in the same namespace.
So we have a problem now. To be honest, I’m struggling with a solution. I can see that the a utility billing table may have a foreign key relationship with an accounting table or a cashiering table. So how can this be modeled across namespaces in .NET classes and still mimic the complex relationship in a database.
If you have not seen WCF REST URL operations or using LINQ against WCF REST then you may want to stop and go learn about this. It’s just unbelievable what capabilities are coming our way. But in order to have this, these technologies are going to have some implementation requirements that may conflict with your or my application namespaces.
Business Entity Objects
Being a “real world” developer today is like standing right over the plate in a batting cage, except without a bat or glove and turning the machine on high. I wish I hand the time to write a Silverlight application that demonstrates this. For now, just use your mind and picture this.
Each of the below phrases is a ball coming at you at high speed. You can’t duck because your boss needs you to implement it. Each ball comes with a collection of 1-4 books hot off the press that you must master. Each time you get hit, you have to figure out how to glue the features and functionality of this ball with previous balls and you must look at the balls in the basket that are about to be thrown at you so that you and somehow plan for what that ball will do to you also. Oh, don’t forget you have to deliver an application to your customer too.
Endless stream of technologies: VS2002 .NET 1, SQL 2000, IIS 5, .NET 1.1, VS2003, .NET 2, VS2005, ASP.NET 2., WinForm 2.0, IIS 6, SQL 2005, .NET 3, WPF, WCF, WF, CardSpace, IIS, .NET 3.5, VS 2008, LINQ, WPF 3.5, ADO.NET (Astoria), SQL 2008, Entity Framework, ASP.NET Futures
So from a code generation perspective we have a problem. Generated class for LINQ can play nice with WPF & WCF. The required attributes can be added by the current generation tools. But what about entities that need to support WPF, WCF, WCF using REST? It makes no sense to have to generate one version of an entity for this and one version of the entity that. A single entity object should be able to work with WPF, WinForm and ASP.NET front ends, be returned by the data layer direct from SQL Server, over a web service, WCF contract or WCF REST call.
Why, because that scenario I just describe is “real world” and I have to do this today in my applications. As time progresses, other API’s and methodologies will be published and our business entities will be required to be portable and consumable.
Currently WCF, WCF REST and LINQ technologies utilize attributes applied to the object, members and methods to describe the object to the technology. This is great, unless the code generation tool does not perform this attributing for you.
This week I did a lot of studying on this topic and found that I got burned by code generators, big time. I had written a Validation, Logging & Form Generation system that was declarative. Meaning that with a few attributes a developer could get a ton of free functionality for their business objects without a single line of code.
Now, with code generators generating entity classes, I can’t apply my attributes any more. Bummer. So I had to rip the system apart and place this code in a partial class method that will get called. It’s not a problem, just that I had lost my ability to attribute my own properties.
Some systems require custom attributes. Take for example the Microsoft Validation Application Block that uses attributes to decorate properties for no code validation. Now that can no longer be used because the developer has no access to the properties to decorate them.
Now that the generator is generating the properties, we can no longer attribute our properties unless we write our own generator or have the ability to supply a template that the Visual Studio 2008 Designers, SqlMetal or SqlTac would utilize. Maybe there is a solution that I have not thought of yet, like dynamically adding attributes at run-time. I would rather not.
While features are being developed faster and faster, these same time savers are also introducing implementation dependencies because more and more code is generated instead of being hand coded by the developer.
Generated Data Access Code
Currently SqlMetal & SqlTac have a big limitation. Again, it boils down to lack of unified metadata and not lack of programming skill. Currently when these products are generating code for a select stored procedure, they both create a new business entity and return that new business entity.
For example take the Select stored procedure GetAllCustomers(). Common sense says that this stored procedure should return all rows in the Customer table and that the business object that called this stored procedure should return a collection of Customer entity objects.
These generation tools will create for you a new entity that has the same exact structure as your Customer object except it won’t be called Customer, the tool will come up with a name for you. This limitation is a show stopper.
With two fields added to the above metadata the problem can be solved. If the Select Returns field is left blank, then allow the generator to make a surrogate class and return that like the tools currently do. If the Select Returns field is filled in then return instances of that class. Same with the SelectAll field. This simply allows developers to assign a return type for their business classes that call stored procedures.
Pulldown’s
For each column the are two fields in the metadata that assist the code generator is generating pull down stored procedures, entity classes and data access code.
If the property or column is the primary key or unique key used in a pull down, then set this field to True, otherwise False.
If the property or column is part of the string that is displayed in the Pulldown, then enter is position number the text will display in the Pulldown text field. If you have only one text field to display then just mark this field as 1. If you have two then mark one as 1 and the other as 2. Example. Our application has a Pulldown for high school on our master address record. The high school Pulldown needs to display the school name and city. So we would mark three properties. The key, school name (1) and city (2). The UI would format the fields as required and would concatenate them as required also.
I’m sure some one will ask, what if my table can be used in two different Pulldown’s. Remember the generation of stored procedures should be a separate step or process from code generation. There will be stored procedures that I want business code generated for that the “code generator” did not write. So for those few tables that you need more than one Pulldown for, just write the stored procedure and the code generator will generate the required code for you.
Templates
A simple published template definition should be publicly available and consumed by Visual Studio 2008 Designers, SqlMetal, SqlTac and other code generators. Why should this be so difficult and not available today?
Templates for generating stored procedures, business entities and business layer entity management code should be available, editable and consumable by the Visual Studio 2008 Designers, SqlMetal and tools like SqlTac.
Stored procedures should be generated first. Then the DBA or developer can add any additional stored procedures they require.
The the tools can generate the business layer classes from the stored procs, tables & metadata.
Need To Work and Deliver Today
One harsh reality that front-line, “real world” developers face every day is that we must deliver products to our customers otherwise we won’t be able to pay our rent. The reason it takes the community years to adopt a new technology is that we just can’t stop the presses and port working production code into a new development framework each time it gets released.
So due to the above limitations and new requirements, I’m being forced to write my own code generation system that has all the above features or get into Code Smith and see if I can use this tool for generation. Lots of work to be sure. Just wish this job was ready done.
I’m hoping that future releases of Visual Studio and all code generation tools will take these points and probably others into consideration. The right hand, left hand and your partners hands really need to start talking to each other, agreeing and implementing solutions that can make developers truly productive and deliver end to end systems including the documentation.
Have a great day!
Just a grain of sand on the worlds beaches.
Posted by Karl Shifflett 
Posted by Karl Shifflett 
Posted by Karl Shifflett 
Code Project VB.NET Article Award Dec 2008
