The longer you wait to shovel the worse it gets, besides the additional snow you have to move. The snow may get compacted, or freeze into ice.  So you will end up breaking ice, salting, besides shoveling. The same is true when maintaining a project, and you wait to fix a troublesome section of code. The problems will start to pile up and dependency may be built on top of the code. Then when you actually get to fixing it you have to fix not only the initial problem, but any dependent code.

Sometimes our hands as developers are tied, the budget maybe tight, but we should understand the business case for fixing something earlier, and try to estimate the cost of not fixing it if there is resistance. That being said we should also be cognizant of balance between our need as programmers for quality, and the bare minimum which will provide our customers, internal or external, the value they expect.

“Fix early, fix often” is my philosophy for dealing with bugs. When you delay fixing a problematic section of code, you will pay for it with  interest.  Having a clean code base is even more satisfying then having driveway and walkway clear of snow.

Bookmark It

The process of the show is fairly straightforward. First the driver may introduce their car, and the contestants may ask them three questions.  Usually something about estimated horse power, if the car has nitrous, or other types of modifications or parts. The contestants make their guesses, and the driver runs down the track, and the person who is the closest gets $100. Repeat until the end of the show, whoever has the most wins. The video below explains it.

To tie this back into software development, and how this is like estimating a feature request. Usually when some one requests a feature, you as the software developer has to come up with an estimate on how long it will take. The requester or an intermediary will come to you and explain the feature, you will have to ask them questions, these questions will help you figure out the basic details about the request.

You like the contestants you must rely on your knowledge and experience to put up a guess on how long it will take.  It is an educated guess. Maybe there will be a driver or mechanical error which could drastically alter actual time. The same is true implementing any feature, maybe their will be programmer error, or an unforeseen complication with the underlying system. Our best estimates will not usually take this into account.

Some people are better at estimation, but a lot of estimation is experience. Usually the first question or thought you have is: “Have you seen something like this, or at least something with similar components.” The more projects you work on, the more you can observe and learn about what slows down or speeds up a project, and use this as a base for estimates. Most quick estimates we make a calculation of a positive and negative factors on some base which is familiar. This is exactly what the contestants are doing.

Bookmark It

As programmers we forget the fact that many of the problems we encounter every day have already been solved. We don’t need to write a sort algorithm, create a new type of data structure, or write an encryption library from scratch.  We try not to repeat ourselves, but we have no problem repeating every one else.

I know that programmers love acronyms, play on words, and guidelines. Therefore, I am proposing the DRAW principal. Don’t Repeat Anyone’s Work.

This is defined as: Not solving generic and common problems that have already been solved. This may take many forms, it could be as simple as using a common library, or it may be more in depth such as reading the code of an open source application, or even looking up a general form of an algorithm.

This is a problem that many programmers have faced for a long time, and we have complained about it for a long time here, here, and here. Sometimes we think we can solve a problem very quickly, sometimes we don’t know some one has already solved it, we may even think we can do a better job, or just maybe we are doing it for hack value. Whatever the reason we have all, took the time to solve a problem that has been solved before, that could have been handled with a quick Google search.

Many times Google will return an answer you are looking for. Last week I had to parse a comma delimited list, passed as a string from from a web service I was integrating into a .NET application. Below is the Google search I used, which return a number of results. Parsing CSV data is something that has been solved thousands of times before. It was that straightforward.

Part of being a good software developer is knowing when to be lazy, and find an already existing solution and when you have to hand code something. The basics of this principal is being knowledgeable about the language you are using, and being able to research problems, and leverage the resources of the developer community at large.

Footnote this is play on the DRY Principle

Bookmark It

Factories in the real world have limits. They only have so many square feet, in which they can fit workers, machines, and resources. Our computers operate under the same principal that we are limited in our resources such as processor speed, RAM, network speed, etc. Part of programming is understanding how to work within these limitations.

Designing and programming applications that operate within the limits requires us to first to know, and then to understand a machines limitations. To know how fast an application will be processed we have to know how much time it will spend reading, writing, and processing.

Moore’s law has greatly alleviated this problem. We now have bigger, faster, and more efficient computers than we could have ever dreamed of, but that doesn’t mean we can ignore the bounds. We still need to understand which operations are expensive, reading data from a network location, compared to relatively cheap operations, such as incrementing an integer.

Even though modern computer hardware has greatly increased the resources we use, there are still areas where we must be vigilant. On example is, a web applications running on a server can easily be brought to their knees by a high user load. When developing web applications with a high usage these principals are not just helpful, they are required.

Once you understand the economics of the machine you can start to make intelligent trade offs for resources. One example is caching, a common strategy for reducing processing and data access by keeping a copy of commonly produced data in memory. These trade offs can have impacts, and create issues. Truly understanding the principals behind machine economies is very important.

Even though we should be aware of these principals, it is important to understand the real world trade off of time spent programming, and the need for software to ship. Many times we will release software that not terribly efficient on purpose. Maybe the time to develop was more pressing than performance, may be it is small web application that has a low usage. Whatever the reason we must accept that sometimes we must release software that is just good enough. That doesn’t mean we can be lazy as programmers, but it means that we should apply the principals of machine economics as we program, and know when we must hand tune our code.

Bookmark It

An example of this would be a QA team is testing a section and they work 4 hours. They spent the 1st hour working and they found 8 bugs, they spend another hour working and only find 4 bugs, after the third hour of testing they found 2 bugs, and in the fourth hour they only find one more bug. We could say that the output diminished each hour they worked.

In the graph below you can see how the number of bugs starts to trail off. This would be an example of how after a certain time the output is decreasing even though more time is spent testing. In this example over 50% of the total bugs found would have been the first hour 80% by the second hour.

Most often the initial QA testing is the most valuable, and then less and less bugs are found the more testing is done. This shapes the way we approach QA, and why many times QA is cut short, because reduction of flaws becomes more and more expensive, especially with larger products. This information also can shape how a testing process is developed, and how deep or shallow we choose to test each section.

Putting this altogether helps us as developers understand that flawless software is very difficult, and expensive to produce, because QA time becomes less and less effective after a certain threshold. The difficult part is figuring out when QA returns are diminished, and is there a point where it starts to drop off drastically? Every project, and every organization are different, and so this could very greatly depending on a number of factors, QA testers ability, software complexity, domain complexity, organizational factors, etc.

The law of diminishing returns as relates to quality and QA ties into the bigger picture of the project triangle (Scope/Quality, Cost, and Time). If we understand how much time it will take to reasonably test our software we can better balance the quality with deadlines and costs. We know that we may be able to shave 20% of a QA testing budget and not cut into the most valuable QA testing time and can more accurately forecast the risk associated with such a decision.

Bookmark It

Programming in assembly requires you to think in bits and bytes. You must allocate your own memory. You have to move data into registers and directly perform arithmetic and logical operations on bits and bytes of data, rather than just defining a for each loop and iterating through objects. Using assembly is in effect directly telling the processor which instructions to execute on which data. You learn how to do the heavy lifting rather than having a high level language do it for you.

Assembly programming can help shore up your knowledge of existing languages. One area where assembly is very effective is understanding memory management. Even though memory management is now handled by modern language with garbage collectors. In assembly we have to explicitly define how much memory we need. We may define 64 bytes for an array, rather than just defining a variable for a list and initializing the object without ever knowing how big that list is. Understanding how memory is allocated can help us minimize the overall foot print of all of our applications and be more efficient.

Beyond this experience with assembly allows you to experience how processing happens. How expensive it is to move things to and from memory and processor. How expensive and slow accessing non memory data such as disk or network resources. It gives you a better feel for bottlenecks in applications. This knowledge becomes extremely valuable when optimizing applications, and understanding how and when to make time and space trade offs.

Overall assembly programming showed me the whole picture of programming from typing in code, compiling the code, and finally running an executable.  This solidified many of the concepts I had learned, how algorithms work and why some are orders of magnitude more efficient, and how data structures are allocated and processed.  These higher level concepts directly improved my knowledge of programming in high level languages, and allowed me to make wiser technical decisions because of it.

Bookmark It

Practice Writing

Above all else, the more you practice writing the better you will be. When I left college my writing was not up to par. It did not improve until I started practicing. My writing may not be perfect, but it has shown vast improvement, because I practiced writing.  Many ways exist for you to practicing writing documentation. Here are just a few that I used.

• Start writing in a blog about technical material you interested in.
• Document your own work. You can use a personal wiki. TiddlyWiki is one example.
• Just sit down and write anything. Just write.

Read Other Documentation

Reading documentation is a way to broaden your knowledge of documentation.  The more you read the better writer you will become. Reading other documentation can help you learn a learn from other’s successes and mistakes. Here are some ideas for reading other documentation.

• The O’Reilly Books great example of technical documentation for Programmers and Administrators.
• Documentation of technical tasks for non-technical users.
• Find similar documentation to the technical item you want to document.

Know Your Audience

Before you ever start writing, ask yourself who is my audience? This sets the stage for the tone of the document and how the information should be presented. Here are a set of questions to clarify who your audience is.

• What technical skills do my users posses?
• What terms will make your users comfortable or feel uneasy?
• What reading level are your users?
• Are they comfortable with technical writing?
• Do they understand the non-technical aspects of the task?

Be Consistent

Keep your documentation consistent. Consistency reduces the barriers to following a complex idea. It increases readers comprehension of the document. Here are some tips on consistency.

• Use consistent jargon ex: If you call a link by the term hyperlink, do not use the term link later.
• If you are adding to a document try to follow the existing writing style.
• Use a consistent writing style.
• Use consistent and proper spelling.
• Use the same font styles consistently.

Write Clearly

There are many ways we can clarify documentation. Technical language is sometimes a necessity, but that does not have to effect the clarity of our writing. Here are some tips for making your documentation slightly more clear.

• Define jargon and acronyms when they are introduced.
• Do not skip steps when documenting tasks.
• Highlight problem areas, potential pitfalls, and common mistakes.
• Provide concrete examples.
• Use the simplest words that work.
• Avoid complex and run on sentences.
• Use paragraphs to isolate larger concepts.

Efficient For the Reader

The only thing worse than incomplete documentation, is inefficient documentation. Writing an efficient document goes hand and hand with clear documentation. Do not make your users search through loads of documentation. Make the important points stick out.

• Good headlines and Sub-Headlines will allow your users to effectively scan content.
• Good section grouping will allow users to find similar concepts quickly.
• A search function for online documentation is extremely effective. I recommend using a Wiki.

Conclusion

These tips will not make you a better writer over night, but they can help you become more cognizant about what you are writing. Below are a list of tips and resources to help with writing documentation.

• The Elements of Style
• Introduction to Technical Writing/Documentation
• OWL (Online Writing Lab at Purdue)
• Software Documentation via Wikipedia
• Key for Effective Documentation

Bookmark It

The programming languages you dine on is based in the same principals as your culinary choices. These decisions are based in your culture. Many factors which contribute to your technical culture: where you learn, where you work, and where you play. Reading these cultural factors can help us understand why a person or an organization picks the languages they do.

Some of us choose us prefer a buffet of languages, and like to sample a little bit of every language. Some choose to stick with a couple core choices and savor every little piece of a language. On the other hand you have your diet specific programmers, such as you free software-gens, and the script-arians.

Sometimes our jobs require us to use a specific language, or the technology requires it. Utility is largest factor in choosing languages. Secondly what you learned and perfected you programming is a close second to choosing which language to use. Tertiary factors are general community factors, such as what are the current trends, what would you like to learn. All of these factors relate to how we choose food, what’s available to us to buy, what our families have eaten, and what we are recommended.

On a side note. The programming languages you choose to dine are signals to other programmers. Just as you culinary choices are signals to other people. These signs, and signals many times are more rooted in stereo type. So many programmers may learn a specific language to affect their appearance to others, just as some people make dining choices to impress other people.

Finally as I discussed earlier the biggest factor in choosing a programming language is the utility, and what we cut our teeth on. Just as many times primary food choices are based on utility, and then what we learned from our families. Understanding these factors can help us grow as programmer, and understand what programming languages we consume.

Bookmark It

The web view of the data has been upgraded to provide more information. The Des Moines Twitter Trends website now provides a clearer view of the data collected. It provides current trends for the past few hours, the past day, and the past week. All of the terms are now links to Twitter Searches.

I added some new information, such as recent links, and current view of the latest Twitter user’s avatars in a grid.

I also created my own theme, and display instead of using a third party Wordpress Theme.

P.S.
Here was the write up for the first version: http://www.dotcodedump.com/2009/07/finding-local-twitter-trends.html

Bookmark It

This primer is broken up into major sections below.

Moving Files

If you are like me your web server isn’t the same machine as your development PC at least I hope not! You will need a way to get files to your web servers.

Push: Push deployments are when files are transferred from the client to the server. An example of this would FTP, or many other file transfer protocol. This involves opening up a client, and manually or automatically transferring the files from your PC to another computer. This could also involve transferring them using a network file system. FTP clients are included with most major operating systems.

Pull: Triggering the server to download files from another location. An example of this would be using a third party program to download the files from web or ftp directory. WGet, SVN, or another similar program could be used.

WGet: Gnu WGet is free software for retrieving files over HTTP

FileZilla: Open source FTP client and server.

Source (Revision) Control

Source control is critical for being able to control releases, and code. Source control allows you to version your applications, and download a particular version from your source control repository. There are many flavors of source control. Personally I prefer Subversion (SVN). For information on source control.

SVN: Open source, source control client and server.

CVS: Open Source, been around since the 80’s, Concurrent Versioning System

Mercurial: Open Source, Distributed Version Control in Python.

GIT: Open Source, Distributed Version Control

Bazaar: Open Source, Distributed Version Control

Visual Source Safe: Microsoft’s Legacy Version Control

Team Foundation Server: Microsoft’s Prefered Version Control, as well as project tracking

Automation

When I first started building websites. All my deployments were done manually over FTP, and it sucked! Then I started using a better FTP client, and guess what it still sucked! I dreaded deploying, it was error prone, and I looked like an idiot because I forgot a file here or there. It all changed when I started automating my deployments, and I haven’t looked back since.

Depending on what language, or platform you are developing on or for. It is very likely there is an automation application for you system. There is a large class of open source build automation applications based in some way off ANT for Java, which is java version of MAKE.

ANT and it’s derivatives are build scripts which are programmed using XML. Using the ANT XML files you can control file copying, running builds, unit tests, and a number of other deployment and build related tasks. Most of these applications support building custom tasks in their native language, so if it doesn’t support what you need it is very easy to build on top of these platforms.

Make: GNU Build Tool, best suited for compiling applications, custom file configuration.

Ant: Java Build Tool, best suited for building and deploying Java applications.

NAnt: .NET port of Ant, best suited for building and deploying .NET applications.

Phing: PHing Is Not GNU make, best suited for deploying PHP applications.

Maven: Similar to Ant, but different application, it is network ready, and uses a system know as the Project Object Model.

Rake: Similar to Make for Ruby, and the config files use a Ruby syntax.

Other Techniques and Tools

Divide and Conqueror

Divide applications deployments up into different sections, so it is very easy to pin point major errors. One example of this is dividing a compiled web applications into binary, and non binary changes, and then deploy one after the other.

Differential

Create a deployment of only the files that have changed since the last release. An example would be using source control to download files that have changed, and only deploying those. The advantages are it will reduce time during deployment. It may also have the issue of missing files, or other items.

Putting it all together

These are just some of the building blocks for deployment strategies. Building a consistent deployment structure will depend on the number of production, and QA environments that host an application. Sometimes you may not even have access to the production servers. These techniques, and tools are very flexible to support even the most rigorous of deployment structures.

The deployment process should never be a source of stress, and deployment processes should never be the source of errors. Using a consistent, and stable deployment process will reduce both stress and errors.

Bookmark It