You Don't Need Tools

High quality software can be created without, advanced tools, automation, and even design patterns. Professionals in many fields besides software development are looking for latest tools, and trends to stay ahead of the competition. Allocating, learning, and implementing new tools can soon become a dangerous cycle, where an emphasis is placed on specific technologies and trends, rather than the product being created. One of the many side effects of using an ever changing tool set is adding entropy and complexity to a development ecosystem.

There are thousands of blogs on software development. Many of these blogs demonstrate, promote, or even proselytize tools or methodologies. The large amount of information provides developers with a large choice of tools and ideas. Searching for proper ideas and tools is very time consuming.

Software developers put knowledge in high regard. Learning is even more valuable. A good software developer should find a balance, between learning and doing. Programming is not philosophy. Making and creating is far more valuable to good programmers. Good is what works, and works well, not what is academically perfect.

Finding tools and even using tools can become a distraction. Learning when your tools are getting in the way is skill, which is underdeveloped, in many programmers. A perfect example is how programmers can get into holy flame wars over a TEXT EDITOR! This is just one example of where the meta-product, becomes more important than the product, but there are many others.

Ironically I am advocating learning how to ignore programming noise, in a blog.

The Lintner Code Complexity Analysis Test: Next time you start incorporating, tools, methodologies, and other ideas into your code base. Ask yourself could an average developer jump in and get the basic flow, and meaning of the code? If no, you may want to rethink your approach.

Personally I advocate simplicity, and minimalism when it makes sense, and some level of complexity is necessary. Tools create complexity. Tools require knowledge to operate, and they will change how your process works. When choosing tools, you must be aware of how they will effect your software development ecosystem over time.

Technologies, tools, ideas, or just plain noise can get in the way of creating, and making. Many times a small project or tasks can be finished the without tools, in the time it would take to set up, and use complex tool. There are many examples of high quality software that was created without code analysis tools, unit tests, and even a structured development methodologies. Next time you start working on a new project, you may want too evaluate your tools too see if they are truly valuable or just wasteful.

Whack A Mole Development

Whack a Mole Development:

When making changes to a section of code, causes other bugs, sometimes recurring, to pop up. I.E. You fix one bug, and three other bugs pop up after you release.

Have you ever worked on code where troubleshooting bugs was like playing a game whack a mole. I have worked on a few applications that seem to have taken this idea to heart.

If you encounter an application that not very stable, many time your options for dealing with it are limited. You just have to work within the constraints of the system. Previously I have mentioned, defensive development as a technique to cope with difficult system. This is an example of controlling what you can, the new code, and fixes you introduce into the system.

When dealing with difficult systems your best weapon is going to be communication. Make everything as clear as possible. Make your new code readable, understandable, and most importantly comment what you changed, and why you needed to change it. Keep documentation on the system. Document critical, and time consuming tasks. If a process took you a week to update, document it, and it will take the next guy half as long.

The most important technique for dealing with difficult systems is asking questions, and making sure you understand what people are asking. Many times "whack a mole" conditions are caused by mis-communication, and lack of knowledge.

Maintaining difficult systems is a reality for many developers. This is as important, or more important of a development skill as creating new applications. Many time maintenance programming is far more difficult than new development, because you have to work outside of your code "comfort zone".

ASP.NET MVC... That was quick!

So I finally delved into ASP.NET MVC. My first impression is it was very easy to get it up and running very quickly. When using the defualt application, I was very easily able to create new routes, and controllers.

The only thing that I did not like was the semantics of constructing a view. It felt so much like the spaghetti code that I tried not to use in ASP.NET. I do see benefit, because one of the down falls with traditional ASP.NET that drove me crazy was trying to implement styling, against the traditional forms controls. I can see very good things from not being tied to those controls, making it easier to implement rich controls such as items from YUI.

My first impression is it's all about the routes. Routes is the magic behind the curtains. Routes control what controllers are run, and how the URL is processed. MVC is a lot more than just URL handling, but coming from ASP.NET forms, good URL handling was one the major things ASP.NET was missing.

MVC is nothing new, but it is new to ASP.NET. I can't wait to delve into it further. I see many places where I previously used ASP.NET forms where MVC would have been a better fit. I like MVC, but I could see it being over-used, and used improperly, where something like ASP.NET forms should have been used. I also can't wait for it to go to full release, then it will be a viable option.

Optimization Is Not Easy

Go to ludicrous speed!

If optimization was easy there would be no slow applications. Slow performance rarely has a single cause. Depending on the number of causes, optimization is a difficult goal. Using a consistent, and defined process needs be a critical part of any optimization effort.

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%" -Donald Knuth

Optimization has a number of side effects, of which we must be aware. During optimization, many times, we must make trade offs. A classic optimization case is, trading processor heavy calculations in favor of using more RAM. Optimizing code can have the unintended effect of making code unreadable, and inflexible.

I mentioned before that Optimization is a difficult goal. I lied. The truth is optimization is not a goal, rather it is a process, also it is best handled as an iterative process. Using an iterative approach allows you to compare your results, and it gives you a better shot at isolating troubled areas. The worst approach you can have is to try tackle everything at once, because you will not be able to correlate the results to a specific change.

My personal process for optimization is pretty straightforward:

• Observe, and Measure
  
• Research
• Change
• Analyze
• Repeat (If Necessary)
  

Once you have a chosen something to optimize the first task is to observe and measure a metric, such as number of operations during a particular action. Attempt to do some research, as to why a particular section of code. Once you have your researched as much as you need to, use your best judgment, and make a change you think will have a positive effect. After making the change analyze whether it performed better than you original measurements, if it did you are on the right path, if not you have just eliminated a potential path.

If you are not happy with the results of a round of iteration. If you continue the process eventually you will notice that you are seeing diminished returns. A few items can be cherry picked, with a simple rewrite of this procedure. If optimizing the easiest items do not achieve your goals, then the difficult work will commence. The same procedure will be followed, but the success rate of the iterations will drop.

Optimization is one of the core group of tasks many developers will need to confront. A consistent, and methodical process will provide a stable foundation to any optimization strategy. Many times optimization goals may not be met, or will require a significant amount of effort. Serious optimization does happen, but we must realize that it is not something that just happens, it requires directed, and methodical effort.

Cloud Computing and the REST of us

Cloud Computing as a buzzword is right up there with social networks, web 2.0, and everything else high technology. I have just begun to delve into researching these services, and I see a lot of potential, but I also have a hesitance that the final incarnation will not look anything like what we see now. I sincerly hope that cloud computing is not just vapor ware (pun intended). It has a possiblity of completly change the way software is hosted, and possibly even architected.

If you work for one the tech giants, or a SF start up your companies may already be creating applications in the cloud. As a web developer not on the bleeding edge this may mean something different. This will most likely serve as an option in the future for hosting larger applications, but I don't think it will wipe out traditional hosting any time soon, just because I believe many companies would be hesitant to put their data out in a "cloud".

As a developer on the ground level I think we will start to see the cloud producing content delivery services, and scalable web services that are reliable and USABLE. Once we have useful services, they can be integrated into our applications without the risk many services have today.

We may even see new languages and programming styles emerge to fit the paradigm, and that will filter back into main stream development. So I probably won't be programming in "C flat++" or "Anaconda", or "IronSaphire" any time soon, but we may see the concepts and ideas filter back to plain old boring VB, and C#.

I believe we will also see a move away from efficient nit picking in development. If you start working in an environment where processor and ram are cheap and on demand, you probably won't want to spend three days optimizing a single sort procedure, becuase it would not yeild any real ROI, and infact probably the opposite.

I do believe that this won't throw traditional programming practices out the window. As a programmer moving into the new realm of highly distributed computing, the fundamentals will be even more important. If a bug exists in single threaded system it only exists once, if a bug exists in a highly distributed system, it could exist a thousand times, and this will require us to be even more dilligent. We will also encounter new types of bugs that exist, because of distributed enviornments.

The discussion has already be started on the bleeding edge of the software community, and it will beging to become more mainstream. The developers on the ground may start to see changes, in how they do things. We will have more options on using third party services in developing their software. We will also be expected to make our applications available as a service for the companies we work with. API style programming will need to become second nature.

This all rehash of the idea that all computers do is input data, output data, and compute data, nothing more. The major change this time around is it will be done on a massive scale. So when I create my widget processing service, it will be able as many widgets as needed, and I never have to worry about machines, and bandwidth, becuase it will now be available, and scale on demand.

Programming Yield Signs

I don't advocate holding any one to any specific sort of rules, but I do have a general set of guidelines that I work with. Here is a simple set of helpful coding warning signs that can save you major coding accidents, and nasty refactorings down the road if you pay attention when you see them. Most of these apply generally to object oriented language, but could apply elsewhere. Without further ado, my programming yield signs.

Recursion

If you are using recursion there may be a better way, or I like to say just because it fits into recursive paradigm, it doesn't mean it HAS to. There are many times I have found myself ending up doing something recursively, and regretting. My word of warning: "It starts with a couple of recursive functions, and then you wake up 3 days later in the middle of New Mexico in playing Russian roulette with Bill Richardson. Recursion has it's uses, but with it you probably are going to incur a performance cost that may not be worth it.

Nesting Logic

Any time you start nesting logic beyond 1 or 2 levels this should be a warning sign that something could be a miss. This is where debugging is generally going to start getting difficult. If you are having difficulty coding a section, and it involves nesting of logic this a point where you need to step back and get a second opinion on how to do it. I am by no means saying don't do it, but watch

Really Long Functions

Writing really long functions is generally a sign of two things, either there is a lot of logic to parse through, or you may be doing something poorly. This may also be an indicator that something else outside of your current scope is amiss. This is generally a point to ask questions, and thoughts on how to program a specific section.

Maze Like Program Flow

If you are constantly searching for functions, and logic, and your program looks like a maze this is probably a point where you should take a step back. This means that either there is some really sloppy code, or there is very high level of systematic complexity. If the complexity does not match the task you are trying to achieve this may a point to step back and take a look at the big picture, and possibly get a different perspective on the situation.

There are many other sets of guidelines, many other people have created. This a small list of warning or "yield" signs that I use regularly. This reflects my general bias for trying to make things as simple as possible, and reducing complexity where ever possible.

Intro to Google App Engine: Displaying Twitter Trends

My adventure into to Twitter land let me to a point. JavaScript alone wasn't going to cut it. I was going to have to go back to the land of the server. I started investigating different options that were preferably free. I settled upon Google App Engine.

The Google App Engine does lock you down to using Python. I took this as a challenge to delve into Python. The App Engine does not use a traditional relational database, instead it uses BigTable built on Google File System. The App Engine is also scalable, and completely hosted by Google.

This is checklist to get an application running on Google App Engine:

• Sign up for an Application (Requires a Google Account)
  
• Download the SDK (Requires Python 2.5)
• Create an application using the template provided in SDK
  
• Upload your application to the AppEngine

Some notes on the App Engine programming, you must use python. You are limited to standard Python 2.5. They provide some extensions, and a limited version of Django. You can upload other "pure Python" extensions.

View my simple Python CGI application for displaying the Twitter trends API call.

My view of the App Engine is it is a service that has potential. The service requires you to download the SDK and work for the command line, so if you are not comfortable with this then it probably is not the service for you. I will probably continue to play with the App Engine, becuase it has some real potential, and I want to learn more about working BigTable.

Once the service is more refined it could be a really smooth way to develop highly scalable web service. Look out for this one as a major player in the cloud market if they decide to take it mainstream with other languages beyond Python.


Here is the source code, also this was my first real dive into Python, beyond just toying with the interpreter (I love it!!!1!)

#!/usr/bin/env python


import wsgiref.handlers


from google.appengine.ext import webapp
#url fetching
from google.appengine.api import urlfetch
from datetime import datetime
#use the simplejson library, lucky it was in django
from django.utils import simplejson
import time

#Output the html ick, but it's just a example
output = '<html><head></head><body>'
output += '<h1>Twitter Trends</h1>'
output += '<ul>'

# Make the API call to twitter
url = "http://search.twitter.com/trends.json"
result = urlfetch.fetch(url)

#Status code ok?
if result.status_code == 200:
json = simplejson.loads(result.content)
#Loop through the trends list
for i in json['trends']:
output += '<li>'
output += '<a href="' + i['url'] + '">'
output += i['name']
output += '</a></li>'

#Finish out the request
output += '</ul>'
output += '</body></html>'

#The primary handler, just outputs output
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write(output)


#Defines the calls through the application
def main():
application = webapp.WSGIApplication([('/', MainHandler)],
                                  debug=True)
wsgiref.handlers.CGIHandler().run(application)


if __name__ == '__main__':
main()