It took me some time to understand the real power behind .Net 2.0 Generics. I think that the lack of usage with delegates hold me back from developing some elegant template-based code. After practicing a little, I managed to pull something off and I thought to share it with you.
My goal:
Create a template for executing an IDbCommand object (as a Reader, NonQuery, Scalar – you name it).
Why?
There is a common pattern for querying the database and that is:
try
{
using (IDbConnection conn = ProviderFactory.CreateConnection())
{
cmd.Connection = conn;
conn.Open();
// use the cmd object.
} //”using” will close the connection even in case of exception.
}
catch (Exception e)
{
// 1. Trace ?
// 2. Rollback transaction ?
// 3. Throw a wrapper exception with some more information ?
}
I want to make this pattern a template so I could trace the entire traffic(sql queries) between my application to my database in one single place !
I can even change the type of the exception it throws or the information it add to it in just one single place !
You got the idea…
Solution:
Step 1 – delegates as generic handlers:
I declare this two delegates:
public delegate T ReaderHandler<T>(IDataReader reader);
public delegate T CommandHandler<T>(IDbCommand cmd);
I believe that they are pretty simple to grasp, lets see the usage of those delegates in the generic template.
Step 2 – The generic “command executer” template:
/// <summary>
/// Simple command executer “design pattern”.
/// </summary>
/// <typeparam name=”T”>The type to return</typeparam>
/// <param name=”cmd”>The command</param>
/// <param name=”handler”>The handler which will receive the open command and handle it (as required)</param>
/// <returns>A generic defined result, according to the handler choice</returns>
public static T ExecuteCommand<T>(IDbCommand cmd, CommandHandler<T> handler) //*1
{
try
{
using (IDbConnection conn = ProviderFactory.CreateConnection()) //*2
{
cmd.Connection = conn;
// Trace the query & parameters.
DatabaseTracer.WriteToTrace(TraceLevel.Verbose, cmd, “Data Access Layer – Query profiler”); //*3
conn.Open();
return handler(cmd); //*4
} //”using” will close the connection even in case of exception.
}
catch (Exception e)
{
// Trace the exception into the same log.
Tracer.WriteToTrace(TraceLevel.Error, e, “Data Access Layer – Exception”); //*5
throw WrapException(e); //*6
}
}
I know, this is hard to watch at first glance, but I’ll walk you through:
*1: Notice the generic type “T” – this will be necessary for returning different types depending on programmer choice (see step 4 for further details).
*2: Create a connection – I’m using some factory I’ve built in order to return a strongly typed connection depending on the selected provider in the application configuration(*.config) file.
*3: Trace the command (CommandText, Parameters etc) – DataBaseTracer class check the “switch” I’ve declared in the .config file and trace the query only if it was requested. This will give me that ability to trace all the queries later on in the production environment (good for prodction debugging).
*4: Send the live command(the connection was opened) to the handler so it can use the command for its needs.
*5: Trace the exception, again, only if requests.
*6: Wrap the exception in DALException – myself, as an architect, believe that the Data Access Layer should throw only DALException exceptions.
Step 3 – A first usage of the template:
/// <summary>
/// Execute the db command as reader and parse it via the given handler.
/// </summary>
/// <typeparam name=”T”>The type to return after parsing the reader.</typeparam>
/// <param name=”cmd”>The command to execute</param>
/// <param name=”handler”>The handler which will parse the reader</param>
/// <returns>A generic defined result, according to the handler choice</returns>
public static T ExecuteReader<T>(IDbCommand cmd, ReaderHandler<T> handler)
{
return DalServices.ExecuteCommand<T>(cmd,
delegate(IDbCommand liveCommand) //*1
{
IDataReader r = liveCommand.ExecuteReader();
return handler(r);
});
}
This one is even harder to follow, but relax, it’s not as bad as you might think.
*1: You can see that I’m using anonymous delegate for the CommandHandler<T>, so the delegate gets the live command object from the ExecuteCommand method and call ExecuteReader() on it. Afterwards, it sends the reader to the ReaderHandler<T> handler (given as parameter).
Step 4 – Real life example, using ExecuteReader<T> to parse a reader into List<Person> and string:
/// <summary>
/// Retrieve the persons according to the specified command.
/// </summary>
/// <param name=”getCmd”>The command to run.</param>
/// <returns>Typed collection of person.</returns>
protected List<Person> ExecuteReader(IDbCommand cmd)
{
return DalServices.ExecuteReader<List<Person>>(cmd,
delegate(IDataReader r)
{
List<Person> persons = new List<Person>();
while (r.Read())
// Create a Person object, fill it by the reader and add it to the “persons” list.
return persons;
});
}
/// <summary>
/// Retrieve the persons xml according to the specified command.
/// </summary>
/// <param name=”getCmd”>The command to run.</param>
/// <returns>Xml representation of the persons.</returns>
protected string ExecuteReader(IDbCommand cmd)
{
return DalServices.ExecuteReader<string>(cmd,
delegate(IDataReader r)
{
StringBuilder res = new StringBuilder();
while (r.Read())
// Build the person object xml and add it to “res”.
return res;
});
}
Step 5 – Leveraging the command executer template:
Now that we understand the template, let’s wrap some more execution “modes”. You can add to it later on, according to your needs.
/// <summary>
/// Execute the db command in “NonQuery mode”.
/// </summary>
/// <param name=”cmd”>The command to parse</param>
/// <returns>Affected rows number</returns>
public static int ExecuteNonQuery(IDbCommand cmd)
{
return DalServices.ExecuteCommand<int>(cmd,
delegate(IDbCommand liveCommand)
{
return liveCommand.ExecuteNonQuery();
});
}
/// <summary>
/// Execute the db command in “Scalar mode”.
/// </summary>
/// <typeparam name=”T”>The type to return after parsing the reader.</typeparam>
/// <param name=”cmd”>The command to execute</param>
/// <returns>A generic defined result, according to the handler choice</returns>
public virtual T ExecuteScalar<T>(IDbCommand cmd)
{
return DalServices.ExecuteCommand<T>(cmd,
delegate(IDbCommand liveCommand)
{
return (T)liveCommand.ExecuteScalar();
});
}
Conclusions:
Now that every command run via ExecuteCommand<T> method I can capture the entire communication between my Application and my database and make some enhanced production debugging, if I’ll need to. In addition, I can decide what to do in case of an exception or add some generic validation checks later on.
All in just one place.
This is a “real life” usage of the power Generics & delegates give us; Using them may be harder at first, but playing with them for a little bit can give you the ability to “patternize” your repetitive code and thus give you a tremendous power to control and manipulate it.
I hope I was clear enough,
Any feedbacks would be great !
Great job, Oren, this is a great example of using the Command Pattern to achieve code reuse. I guess we can write infrustructure better than MS (or at least more specific to our needs) if we want to… MSDN would provide you with the code sample and have you repeat it all over your application, instead of simply providing a usable and reusable piece of code like this one.
I still predict that if you continue in this direction you will soon eliminate most of the code generation you currently use in your applications. :-)
This is a nice wrapper, but may I ask why do you handle this yourself?
There great frameworks out there that do it for you automatically.
In the case of your List<Person> reader, assuming that you’ve the following table structure:
People:
– Id
– Name
– Email
– – Many More
[ActiveRecord("People")]
public class Person : ActiveRecordBase<Person>
{
string name,email
int id;
[PrimaryKey(PrimaryKeyType.Identity, Access=PropertyAccess.NoSetterCamelCase)]
public int Id { get { return id;}
[Property]
public string Name { get { return name ; } set { name = value;} }
[Property]
public string Email { get { return email; } set { email = value; } }
// define more properties here
// define methods here
}
And then you can do:
foreach(Person jobLess in Person.Find(Expression.IsNull("Job")))
{
OfferJob(jobLess);
}
You also get CRUD operations for free.
p.s: I couldn’t submit because I’d <, > in the request. I gues I shouldn’t complain because my blog does the same :-), but AFAIK the encoding is handled by dasBlog.
I Like this pattern, but I have two problems:
1) Not all programmers know how to use delegates, events as needed, you even mention this in one of your posts.
and that should cause bugs, because mediocre programmer right bugs with no specific reason(I like this one:).
mainly, when the code you read is harder to trace.
2) This idea is much harder to implement in the .net 1.1, without the generic.
personally,
I’ll start implement it… ;)
I’ll tell you if i’ll get further desgin remarks.
First of all, thanks for the feedbacks everyone !
Ayende – (1) I would like to write this pieces of code to improve my skills and practice the new features of .Net 2.0. (2) I don’t use ActiveRecords nor NHibernite – I’m generating my code with an engine I’ve built, and I’m quite happy with the results, but thanks for mentioning those tools. Combining my code generation with solid infrastructure seems to me as a good path for better solutions.
about the < > -> this is a well known problem my friend, I wonder if dasBlog will support it on the next version (1.9, coming soon).
Yoni & Shani – Yoni, you must keep in mind that Microsoft release code for the mediocre programmers and this piece of code isn’t so simple to understand and use (as Shani mentioned). I do agree that we can write some better suited code ourselfs sometimes, but I think that the most important part is to well document and explain(\teach) it to others so as one person (the "genius" that wrote the infrastructure) quits we can still change the infrastructure by our needs and keep moving forward.
BUT(!) I don’t agree that this excuse, of "mediocre programmers", should cause us to write mediocre code, just so other programmers in our team\department\company would have "easier life", I see 3 solutions:
(1) Teach. I believe that a good 1-1.5 hour lecture can cover delegeates and Generics so this code will be a little clearer.
(2) Let them practice. Write a good exercise so your programmers can practice in writing delegates and some simple Generic methods\classes.
(3) Code Review. Like a good chef, you have the responsibility to release good code(application) to your clients. Now it’s the time to catch poor code.
btw –
Yoni, you know that we "argue" about whether code generation is good or bad (.vs. solid infratstructure), I believe that as long as you don’t have good knowledge about how to write good infrastructure, code genereation or ORM is a great way to move forward. If I have now 80% generated code and 20% infrastructures, I truely believe that in 1-2 years from now I’ll have 40%-60%. But this is the path I would like to take. I see a lot of companies with "infrastructure team(s)" which are made from a bunch of folks without any real experience in developing applications and I see their results. I believe in "natrual growing". A good infrastructure team is built from people with experience (at least 5 years, IMHO) and abilities to produce great code. I’m getting there :)
As for the average programmer, I think a simple change in the naming of this function would help a lot (it’ll also help the above average programmers).
I would rename it to something like:
public static AndReturnsThisType ExecuteCodeBlockWhichUsesIDbCommand < AndReturnsThisType > (CommandHandler < AndReturnsThisType > theCodeBlock, IDbCommand theCommandItNeeds){…}
Maybe using the word ‘Delegate’ instead of ‘CodeBlock’ would be better but this is the general idea…
Between intellisense and compile time errors the programmer should get the picture. I’m assuming the programmer knows what Generics and Delgates are (or else he/she will have a hard time with .NET collections and event mechanism as well).
I think you got it right that infrastructure programming should never be disconnected from actual application programming. I can easily tell that the code you showed us in this entry was a result of duplication noticed by you in the development cycle and that is what makes it so usable and extendable. This process of Refactoring should be done on a daily basis by both beginners and expert programmers.
As for code generation, I think its fairly safe to say that refactoring is easier for the average programmer than writing code generators… So maybe if the people writing the code generators instead help the people writing the application refactor their code more often, better infrastructure will arise and less code will be generated.
I agree with Shani that this could not have been done in this manner – which is strongly typed – without Generics. That is why I mentioned that, knowing your current framework is strongly typed, Generics will help you eliminate a lot of generated code, which was originally generated to achieve strong typing more than anything else.
As for people like me who never use strong typing, Generics doesn’t help with anything… I’ve already written something like this without generics in a project I worked on a while ago, of course it was weakly typed.
Yoni: "As for code generation, I think its fairly safe to say that refactoring is easier for the average programmer than writing code generators… So maybe if the people writing the code generators instead help the people writing the application refactor their code more often, better infrastructure will arise and less code will be generated. "
Good point. I’ll have to sleep on it for the next few months, maybe an inner change will arise and that’s the path I’ll take. I trust my instenicts as I’m progressing in the development world, and the environment (people around me, the given application I develop, pressure from management, my state of mind etc) has its impact on my decsisions. For now, I feel that code generation + good solid infrastructure is more pragramtic than daily refactor.
See, in order to develop good infrastructure, you need good programmers and extra time or big projects(more than ~6 months of development) so you’ll have to proper time for (1)defining the "problems", (2)think about and offer "solutions", (3) DR(design review) and of course – (4)implement and (5)QA them. Company like Ness, for example, has the ability to work on great infrastructure in terms of time as their projects are quite long, but I don’t have this(time) luxary right now.
I must be pragmattic with the short time I have, which means I’ll have to do phases 1,2,3 almost always on my "extra" time – this isn’t an healthy path, especially for building a solid infrastructure.
That’s is my world – for now.
Oren, I think you made a clear statement of your intentions in your blog’s title – "Striving for Agile Development".
Moving to Agile is a transition requiring good programmers as you’ve mentioned. Extra time and big projects are not really required because Agile is not about spending time and resources on infrastructure at all, and is said to be more effective in small-to-medium-sized projects.
I think we are still lacking a rock solid framework for online transaction processing applications but I sure don’t expect a busy development team to produce one while doing other things. Agile is about developing mission specific infrastructure as a part of the development process. Methodologies such as eXtreme Programming and Test Driven Development lead to developing in small iterations, with continuous integration and a lot of Refactoring. This means eliminating a lot of the work usually done beforehand (steps 1,2,3) and incorporating it into the actual coding (leaving you your "extra" time to do more important things…).
The bigger issues with turning Agile these days are management issues but lets not get into that…
Without much knowledge of either companies I would guess SQLink has a much bigger chance of adopting some Agile Development ideas than Ness. If I were you I would get the developers Fowler’s ‘Refactoring’ (or maybe a lighter extention book on refactoring) to read at home. Design Patterns will come next and then perhaps TDD… All of which will do them a lot more good than the next ADO.NET-how-to-fetch-a-row-in-20-code-lines…