Development or Discovery

The biggest cost in any software development effort is always a factor of time.  It became pretty obvious to me while working on a new reporting project.  I've worked on a "ton" of systems over the past few years and with very few exceptions I can sit down and start making tangible progress very quickly (this may not be evident to my clients while I'm working on the plumbing, but that's a topic for another post) but with this project I've got about 4-6 hours into it and although I'm making headway on a couple of fronts I'm starting to sense the "churn" where it's more of a discovery process than a development effort.  How much of your time is spent in discovery and how much in pure development?  I think the ratio of discovery to development is major factor in developer efficiency and thus the time/cost in a software development effort.  Although some external factors such as training, pair programming, documentation and mature processes may help with this ratio, the biggest impact on the ratio will be the person doing the development.

Discovery does not provide any real business value.  Discovery is a critical investment that needs to be made.  Just as you invest in your financial future, you should invest in your professional one as well.  Once you’ve made this investment you start building capital, the capital can then be used to increase your Development time, this is where your investments should pay off.  As Frederick Brooks states in his timeless classic “The difference between an exceptional developer and a Mediocre one is generally an order of magnitude”.  Why is this the case?  One reason is Developer Efficiency  but the other is the capital the developer brings to the effort.  Effective program manager realize this.  When we talk about capital/experience/knowledge this can really be in three categories, general technology, domain specific, implementation specific.

Technology Capital

You, my dear reader are 100% responsible for this.  If you work for a corporation that’s nice in that they may pay you to learn this knowledge, but in the end, this is how to program an Http Handler in ASP.NET or write a SQL Query, create a clustered index and so on.  These skills are not implementation specific.  The wider background of skill sets you have the more capital you will have and the more value you bring.  As a consultant, if I am stuck doing “discovery” on a new technology I very rarely bill the client for this time.  My clients pay me to “develop” they assume I know they technology.

Domain Specific

This is where you can start developing additional capital.  Unless you pick on specific vertical, there will always be a portion of the time working on development efforts where you will be doing discovery within that domain.  An example of this is my experience working in a Sales Force Automation company for 3 years on a number of different efforts.  Although there was an initial learning curve in this vertical, over time the value I brought to the table during the system design meetings increased, the more knowledge I brought to the table, the quicker we could talk about what made their problems unique and in most cases these so-called “unique” problems were actually solved in other efforts and that experience could be capitalized on.  My goal going forward is to pick a new domain every other year and start gaining that knowledge.  Have you worked in one industry all your life?  How many jobs are there in that industry?  How well are you compensated in your industry with respect to others?

Client/Implementation Specific

Now is where it get’s interesting, each client does indeed have certain technologies, systems, database schemas that are unique to their organization.  I would suspect that a considerable number of developers are “experts” in this area.  As a consultant I worked at one client for about 3 or 4 years.  They made a considerable investment in me in their specific data, not only just the structure, tables but also the subject matter.  I made a significant investment in time in their data.  At the point I started scaling back my work on their efforts, I had a much better understanding of the data within their systems then anyone did internally.  This was a great engagement for both parties (Client & Software Logistics) however when I look back over those 3 or 4 years, what can I take away from that client specific data knowledge?  Not very much as it turns out.  From a billing perspective, I don’t have any problem doing “discovery” of a clients systems on their “nickel” especially if in the long run that knowledge won’t be useful anywhere but their systems.

The bottom line is in software development, the more capital you bring to the table, the less time you are doing discovery and the more time you do performing development.  You are paid by your organization as an employee or a consultant to do development and not discovery.

-ec

Putting your Trust in Metrics

I'm a big fan of automating as much as you can, but blindly automating processes controlled by metrics may cause more problems than they solve.  The idea behind capturing metrics or in the case I'm currently working on, writing a suite of unit tests is to automate the testing so I can make changes at-will, run my unit test and know everything is still working good.  If I come to rely on these unit tests (or really any sort of metrics) to help me make business decisions as to when I am code complete and ready to ship;unless I'm 100% sure that the unit tests take into account 99.99% of all the edge cases; I'm doing the recipients of the software a disservice.  As an efficient coder, I want to take what the customer tells me they need and turn it into the compiled bits that successfully solve their problem in a straight of a line as possible.  Unit testing and other software related automated processes go a long ways to helping ensure an efficient path, but as these automated processes are developed we need to pay special attention to make sure these are complete.  In a large sense, if you are capturing KPI or Key Performance Indicators about your organizations performance, are you sure that the core measures you capture are indeed complete and accurate?  Remember the purpose of capturing these metrics is to influence business decisions.  If your metrics are not accurate or your measures just can't be trusted, there are three core problems that this causes.  First it clouds the judgment of the people directing the efforts, they may think they need to do A, but the metrics tell them B really needs to happen, A is clearly the right thing to do, but since B is out there as a potential solution it may dilute the affect of implementing A.  Second it puts in place a false sense that you are indeed doing something to solve your problems and all you are really doing is delaying the true solution costing the most valuable asset in software development, time.  Finally it takes time to capture metrics and if done properly it is a great tool.  If your metrics are worthless you waste the peoples time building the infrastructure to capture the metrics, you waste the peoples time to capture the metrics and you may even unintentionally sabotage future metrics capturing initiatives.

In a previous post I discussed the race car analogy, where the drivers have a significant amount of safety gear in their cars.  They need to be 99.99% sure that if they get in an accident the gear will do it's job and protect them.  If they can rely on their gear, they can take more risks to give themselves a competitive edge.  Are your sure the test cases, scenarios and metrics your are employing would help you prevent or survive a crash and give you a competitive edge?

-ec

Developer Efficiency

At the very essence of what we do as technical developer folk is take the needs of our business stake holders and turn those into 1's and 0's (and maybe in our lifetime we might figure out how to get 2 ) that business stake holder can run as software on our computers.  Any thing else is pure rubbish! 

So what are the attributes of an efficient coder?

Technical Aptitude

This is a baseline of becoming an efficient coder, when it's all said and done you must have a certain level of Technical Aptitude, that's not to say you need to be a "Rocket Scientist" however you should have certain skill such as problem solving, a general nature to ask "hmmm...I wonder how that actually works?" or "boy that sure is some cool technology, I wonder if I could build that?".  I think this is something you either have or you don't.  Not having it is not a bad thing it just means this type of work may not be a good fit.  I know any time I try to do something creative with any sort of graphics problem, I can spend 3-4 hours on trying new things and in the end it probably looks worse that when I started.  This isn't a bad thing, but the sooner you can recognize your weaknesses, the sooner you can find other people around that complement your strengths that make up for those weaknesses.

Memory

As a developer there are just too many different technologies that could be implemented to solve a problem, too many Languages and API's to learn, too many object models, too many command prompt arguments and keywords, you get the idea.  The more of these you can stash away and bring to the fore-front when necessary the quicker you can continue developing and not doing discovery.

Perceptiveness

Being able to read between the lines and think to a long term strategy is important when developing your solutions.  I see Software Development as chess match.  At any given time you are presented with a certain set of facts or knowledge, each time you sit down to design/code the moves you make have a major impact in how the pieces get moved and sets you up for your next designing/coding session.  I've just seen too many software efforts take the fundamentally wrong approach early on, and those projects never really recover.  Although an extreme example, let's say you wanted to write a simple contact manager, do you think you would start writing this in Assembler?  How about some sort of auditing function where a copy of all items was made into a database every 15 minutes, then a process was run to detect and report on changes, this was a real world scenario where a developer worked on getting this process working for about four weeks, and never did...the efficient coder made the correct chess moves and had implemented in an afternoon.  I see this is a very common problem in software development.  Do you start coding before you have a true understanding of the problem you are trying to solve?

Attention to Detail

This is a really hard one for me, coming up with creative solutions and looking at the big picture sometimes blurs some of the details.  What's interesting is that when I'm doing low-level programming such as embedded systems development, my eye for detail is very sharp...go figure...to be a efficient coder means developing system with little or no bugs.  The challenge here is that if you pay too much attention to the details, you may loose site of the big picture.  Coding something that implements the fundamentally wrong solution even if perfect and bug free is worth little to no value.  I think this is one of the bigger areas where we can use technologies, tool and processes to improve our efficiency.  After all computers aren't very good at creativity however they are extremely good at the details.

Experience Level

I think by far the biggest impact on this ratio is the experience of the person doing the development.  This experience is not just in the actual programming languages such as Java, C, C++, or the Software Development Life Cycle/methodologies, but to an even greater extent programming a wide variety of business processes.  Just as with software, certain things can be "abstracted" the same is true with business processes.  This experience also helps facilitate the conversations with the business stake holders, the more you understand about business in general, the more time you can more quickly get to the specifics of what makes this problem unique. 

Experience only comes with time.  You can do certain things that may not seem that important, but really are.  There is no substitute for spending time with the business stake holders even is you just sit in the room and not contribute anything.  That is not to say they should have a direct line or to influence your day-to-day scheduling.  But as a manager early on, I used any possible opportunity to get the developers in a room with the stake holder.  When doing embedded systems work it was always a good reminder going to the field and realizing that in the real world when my software is working it's not just a little LED that comes on at the right time but it's actually a 20 ton rock crusher that would be very bad if it came on at the wrong time.  It puts everything you do in a different perspective.  How often do you have actual conversations with your business stake holders?

Passion

What really differentiates you as an "efficient coder"?  Do you really just plain and simple love what you do?  Or are you just coding to pick up a paycheck?  Life is just too short to spend the majority of it working in a job just to pickup a paycheck.  If at a certain extent you have qualities mentioned above and your are passionate about taking the needs of the business stake holders and turning them into the 1's and 0's in exactly the right order as efficiently as possible (what ever that means on your particular effort) the chances of you being an "efficient coder" are pretty good!

Are you an "efficient coder"?

- ec

How can you tell how efficient your software development/migration process is?

It might be interesting to develop a set of metrics that capture the amount of time and steps involved with doing certain common activitives on a development effort.  A simple but I think effective measurement might be what is involved with adding a data-driven, context sensitive drop down list to a WinForm or WebForm.  This would involve steps similar to the following:

• Adding the data source to populate the drop down
• Adding the field to be controlled to the data base
• Getting the source from your source control repository
• Modifying the data transfer objects to get the selected value between the database and the control
• Modify the user interface to add the control
• Checking in the changes
• Add some sort of help documentation
• Develop any required tests
• Migration to the three typical environments (Dev, Stage, Prod) for both the application code and database objects.

Come to think of it if this process was captured (even without any metrics) it would go a long ways for developer documentation.

I recently came across an interesting tool for looking at source code metrics SourceMonitor from Campwood Software.  This tool looks at source within a set of sub-directories and gives you basic measures about your software.

The following are a sample of measures captured through this application:

• Total Source Lines
• Total Statements
• Percentage of documentation
• Complexity
• Block Depth

A very interesting chart is the summary one below; this gives you an idea of how well your source fits within a predefined set of measures.