I was (am) listening to the Alt.Net podcasts today.  I just came across this podcast series last night.  I don't know why I didn't hear of this before, but perhaps it's because it's relatively new as it's only been publishing for a few months and is up to 8 episodes.

At the same time, I'm on a three hour train ride from Baltimore up to Connecticut to talk to a potential new client (to be specific, an interview).  The combination of brushing up on some things I don't do on a daily basis in case I'm asked about them, as well as some concepts discussed in this broadcast has brought me around to something that's on my mind relatively often but that I've never written down...

What are the key concepts in software engineering, as of today, in July 2008?  This is not meant to be an complete list, or that the items exhibit complete separation of concerns relative to each other.

Here my initial stab at what I think are the things you must exhibit to be an exceptional architect at this time:

• Domain Driven Design (DDD)
• Test Driven Design (TDD)
  
• Software Evolution / Continuous Improvement
• Patterns, Patterns, Patterns
• Dependency Injection and Inversion of Control
• Policy based programming
• Aspect Oriented Programming
  
• Doing things as much in a declarative nature as possible
  
• Domain Specific Languages
  
• Separation of concerns
• Design by contract / Interfaces
• Service Orientation
  
• Transparent translation of domain to persistent store
• Mocking
• Parallelism
• Caching
• Factories
• POCO 
• Event Driven Design
• Orchestration
• Rules
• Logging / Instrumentation
• RDBMS design
  
• Keep it simple stupid
  

I'm going to come back and expand this list as well as elaborate on each.

I was thinking about this question today.  I came up with this answer.  Probably better for a tweet, but I thought I'd write it here anyway:

I Love to learn, create, and through those provide services that allow others to exceed their goals

I'm brushing up on my LInQ today for a few reasons.  One is that its been a while since I used it because my current client isn't that far ahead and I need a refresher.  Second is that I want to build up some interview questions.  Third is I need to update an intro to LInQ presentation I have previously built. 

The true reason though is that I want to write out a few of my opinions on the technology, and explain a few things from the perspective of how I see them.  One of those things is how the new C# 3.0 language extensions facilitate LInQ.  I think sometimes that people think that these features were added to C# as independent items that developers have requested.  I don't really see it that way, as I think that although they are useful (and very useful at that) by themselves, that they are only added to the language to allow LInQ.  If they were not added, the expressivness of LInQ statements would be greatly reduced, and as a matter of fact it would not event be possible.

The following are the new C# 3.0 features and how I see them related to LInQ:

• Local Type Inference

Required for handling query / selection results.  Often (mostly) the type of this object may not be of a type that is defined in the system at compile time.  An example is a projection of just a few properties from a result object (via select) instead of the whole object.  In this case, the compiler / run-time will create an anonymous type whose type (likely a class with with properties) that can be inferred through the a var keyword.  This is tightly intertwined with anonymous types.

• Anonymous Types

Required to allow the creation of results from where and select queries.

• Lambda Expressions

Required to efficiently code expressions for the selection and projection steps of a query.  With out them, you would need to declare a delegate (either explicitly or anonymously) which would add to the coding overhead.

• Extension Methods

Needed to extend existing classes to have new LInQ methods such as .Where, .OrderBy, .Select, ...

• Object Initializers

Allow the compiler / run-time to efficiently initialize properties of anonymous classes.  Without them a constructor would need to be injected into the anonymous class.

• Query Expressions

Provide a syntax similar to SQL, which is used to manipulate data, but provides a short-hand for the coder that the compiler converts into specific classes and method in LInQ.

In FoP, Karl states that your goal is to create highly maintainable code.  His thesis has many presecriptions for this, but I wanted to make a few comments about what I consider the process of creating.

I program computers because I like to create.  I like to "design" new things, and make them reality.  Basically, I like to take things that already exist, and apply the things that I have learned to make them do new things.  To be technical, as "Herbert Simon" would say in "The Sciences of the Artificial", what I do in designing things is apply knowledge to transform something into something else; I create through the process of design, which is an artifical process because it uses cognition instead of being naturally occurring.

Now, this creating can be done done several ways.  One is to modify something that already exists to do something differently.  Another is to take two or more things that already exist, and combine them together with some new stuff to make them work together to do something new.  Or you can just create something altogether new.

Regardless of which of these scenarios, the end result is something that I want to be proud of.  And I like Karl's statement that your goal as a programmer should be to make highly maintainable code, and I think this is spot on.  That is how a programmer has his achievment most accurately measured.

Why?

Because showing that I created something new, and did it well, is what gives me the most satisfaction.  Highly maintanable code means that the code can be understood, that it follows patterns of practice that have been gained through my experiences, and probably most importantly that because of both of these the people using it can learn to do the same if not better.

But what is meant by "highly maintainable code"?

I think this breaks down to several things:

1. The code is built using patterns of practice that are easily understandable so that it can be "maintained" - that is understood completely by others,
2. The code is tested and proven to do what it is intended to do, otherwhise it is surely not "maintainable".

I used to be skeptical of TDD.  I now embrace it wholly.  I used to wonder why I made this change.  And I quess that is why I was so perturbed in a previous post when I mentioned that I was actively prevented from building a test.  After all, without being able to create the test and prove what you did, how can you tell if you created something effectively?  And what better a way of defining the measurement of your quality at the start so that you can guide your way to your goal?

 

I found this "book" yesterday after Scott Hansleman twittered about it.  I just started reading it and just a few pages in I already felt inspired to start discussing it.  This is purely based upon some initial statements made about YAGNI (You Aren't Going to Need It) and being in the ALT.NET frame of mind instead of the MSDN frame of mind.

I really enjoyed the initial statements on MSDN versus ALT.NET.  I really do feel that there is a big difference between these ways of doing things.  In the book, Karl quotes Jeremy Miller (who I used to work with) as stating "that the MSDN way is a practice of having too much focus on learning the API and framework details and not enough emphasis on design and coding fundamentals."

I absolutely concur 1000% with this statement.  I used to think this way, because that was, for many years, what I was told was the way to think.  There used to be a days when I used to know the entire Win32 API, and I was really proud of it.

Then one day I realized that this wasn't doing me very much good; it wasn't necessarily doing me much harm, but it was limiting me in that through the process of this type of learning I was unknowingly preventing myself from really seeing how to solve real issues in software that were important.

I don't know where or when it happened (but I'll guess is was about 10 years ago if not 5 or so years before that), I started to think in larger abstractions about the problems I was dealing with.  I started to think that, well, in my adventures in enterprise software development, that:

1. I wasn't really solving any new problems
2. All of this detailed knowledge wasn't helping me solve those problems any better every time I had to do it again
3. That there must be better ways of doing this

I don't know, perhaps you might say that was when the lightbulb turned on and I started to thing like an architect instead of a programmer.  That may be it, but I think its too simple of a statement.  And I'm not sure what I think about the moniker "architect" either, as I think today it presupposes things, both good and bad, that may not really be important either.

I guess what I'm saying is that after years of trying to solve how to do things better, it feels that it is to me exactly what Karl states about the philosophy of ALT.NET, which is to focus on alternate solutions and approaches to the MSDN way of thinking, and I'll add specifically that this is a way of using best practices learned through experience of the community to solve problems through knowing the patterns of how to solve the problems, and then worrying about the details of solving the problem only if there isn't a solution already available.

To me this is a very fundamentally different way of thinking about going about the things that I do.  I often get asked many times what are the specific method or so and so class in interviews, and I'm often like, "I don't know off my head, but give me a second with Google, or better yet, why know them when I can solve the problem with a better tool providing a solution to the overall problem."

See, I think this kind of stuff is just cognitive overhead, knowing all these details.  Over the years, instead of knowing all the details in memory, I've built ways of knowing how to look up the details at a moments notice.  And by defering to other tools (like Google, or open source projects, or patterns and practices) for storage of the details, I really feel like I have allowed myself to become more creative and at the same time more productive.  I've effectively become more atune to being able to solve the problem effectively, instead of just focusing on mastering the tools.

The thing that I wonder is if this type of reasoning can be taught from inception instead of having to learn through street smarts over many years.  I think that sometimes this can be, but at other times I think we're just so entrenched in software with mastering tools instead of mastering solutions using the tools.

I mean it still amazes me that in many places that I consult, or with people that I talk with in my field, that they just stick with this means of doing things the way it always has been done.  Its really an anti-pattern, and it's a difficult one for many to break.

For example, I once told another developer on a project that I was consulting for, in front of his boss, that when he was done with some code he was writing to integrate data between several systems, that I would like to write a unit test for this code.  This would be something that would be mission critical, and run every day, and I wanted to make sure that if we had problems with it that I could be able to prove the failure using the test so as to fix it and make sure we don't have that problem again.

Well, I was aghast by the reponse I got, almost before I even got the entire sentence out.  Testing wasn't important, and that I need not (and I mean that I am flat out not to) build that test, as there were more important things for me to focus on.

Well, the really amazing thing about this was that this piece of mission criticality was just run for the first time, with some dummy data, and that was apparently good enough to push it into production the next week without any additional testing or plans on how to handle it if it failed.

Guess what?  It failed.  And it failed hard the next week.  And the users were pissed.  And I got to write that test then.  And it was another 8 weeks until the users allowed us to try this again, delaying us several months when I could have put together preventative measures in a few hours of work.

So, where am I going with this?  Perhaps nowhere really, but it's stuff kicked into my mind in the first few pages of this book that I though I had to get out.  I do really think that we're moving into a new paradigm in the next 10 years with software, and I think that some of this is embodied with this contrast of MSDN and ALT.NET.  I just hope we as a community can make this happen.

Some notes on this podcast that I think are of importance...

Ted states that event based programming differs from event driven systems.  An example of an event driven app is a windows GUI.  These event driven programs do nothing until receiving event and then handle it internally.  Event-based applications, on the other hand, basically operate upon upon internal communications between different parts of the application that are hooked together to provide functionality through event based communications. 

Dynamic Routing is stated as being one of the important things about this type of programming.  Dynamic routing is where handlers for events can be changed at run-time.  In event based systems this is fixed at runtime.  Event-based applications can change the configuration on the fly.  To do this, several constructs are used:

• Builders - instantiate all the objects in the system
• Binders - provide communications channels between systems
• Coordinators - provide orchestration between objects through binders

The key point is that binders preserve the decoupling of sources of events from their listeners.  This is important to be able to ensure flexibility, independent development, and independent testing of the components.  A builder puts together all entities that will be wired, and then passes control to a binder which wires the individual components.  Binders then wire up all the observers to subjects. 

description of switch statements being used as a means for demultiplexing data moved over a single communications channel.  At some point some data is created, then through one code path (the communications channel) a decision is made to go different ways.  This inherently bad for concurrency and scalability, and handling through bindings is much more scalable.

Also raised was a concern about the .NET/WPF model of events and passing argugments to the event through classes as a parameter.  To create your own events, you must necessarily then create an event arguments class.  This therefore causes class explosion as you need to make a new class for every event just to pass the arguments.  I totally agree with this as I think its a total pain to have to go through this.

Ted has a book on this topic.  I have not read it yet: Event-Based Programming: Taking Events to the Limit.

I'm curious to see if he gets into topics of Invasive Software Composition.  I recommend the book on this by Uwe Assmann.  This book discusses a number of topics, but which I would describe as mostly discussing AoP and connection of software components through connectors, ports and adapters.  It's also a good read if you are into SOA and BPEL, as (at least to me) it gave good clarification on the use of communications channels (SOAP/REST) between components (Web Services) and transformation (XSLT) of data (XML) across the channel.

Where this ties into the discussion that Ted had is that Assmann (boy that's a bad name in the US) would do the "binding" with AoP constructs, basically injecting the bindings between the components at runtime through dependency injection (DI).  I would also tend to say that the bindings could not also be done this way, but also through generic programming using policy classes, as well as havnig routes between objects be able to be modified dynamically through having the injected policies either change or make decisions on the data.

This last point gets into another SOA tennant (a reason I brought it up earlier)  which is know as content based routing.  Typically, this is done on an enterprise service bus through logic attached to the bus, but within an application, a very flexible model of routing of data as well as code flow could be set up with a good combination of AoP, DI and event based architecture.

After several months of issues getting registered for it through Sylvan Prometric, I'm finally in and up and going on for the Microsoft Certified Architect certification, focusing on Solutions Architecture.  I'm not exactly sure where this will go, but I'll write about it during my journey.