Separating out the source and the configuration files

One of the first steps before delving deep into the coding is to make a clear distinction between the source and the configuration files. Every software must give enough options to its users to configure the software behavior. A user may wish to change the interface color, rearrange certain options and reposition certain elements. Sometimes giving this additional power to the users to tweak the softwares is seen as challenging. This is the prime reason that many softwares that allow the users to configure their softwares often give an option “Restore to the Default values”. There is this common fear among the programmers that the naive users will tweak the software in a manner that will make the software unusable.

Another possible reason is that the programmers often tend to hard code all the possible configurations in their programs. This is mostly because of the ease in programming. Instead of reading a value from the configuration file every time an action must be performed, it is easier to work with the hard coded values. And of course, there is an associated delay in reading the configuration value. So most programmers tend to work with values hard coded into their programs.

And then when there is a demand for a change (for example color), they make the change in the code and release a new version. I agree that there is often no clear distinction between what must be available to the user for configuration and what must not be not. And personally I found very less books and papers that talk on the topic. But it can be concluded that ease of programming and fear of software damage are the prime reasons why programmers do not give enough configurable options to their users.

The caveat associated this non-separation is that the programmers often spend a lot of time tweaking with various values. They have to yield to the popular demand. Often, it results in the loss of some potential customers who are not satisfied with the change and move on with other softwares available in the market.

Sometimes it is not intentionally done. Take for example in mission critical systems where every fraction of second counts and the values chosen are made after enough careful research.

To make a clear distinction between what must be configurable and what must be not at the time of development is a challenge in itself. On one hand, you can make every possible option to be configurable, to an extent that you can give the users the choice between algorithms (for example bubble sort, quick sort for sorting). Such an approach seems to be laughable, but an expert knows what is the best algorithm for his/her requirements. And on the other hand, you can hard code, leaving no option for configuration. If you are designing a generic software, there seems to be no perfect option. There must be enough options for the user to configure their software.

Here is what I feel every software development should look like hypothetically

$ ls project
src config

Two separate directories (or some other means of separation) for source code and configuration

Piping: A simple means of data transfer between commands and programs

When we were discussing about Data processing, we saw that a data processing machine consists of a number of data processors working in unison (in a series or in parallel) to generate a meaningful output. To see such a data processing in action, we can take a look at the idea of piping in Linux (or Unix). A pipe reminds me of the cylindrical tubes that are used to transfer water or oil from one place to another. So a pipe is something which doesn’t modify the data (or entity) passed through it, rather it is just for transfer. In other words, the input to a pipe is the same as the output.

When we discussed about commands, we saw that there was a time, when the users had to enter the name of a command on the terminal of a computer to get things done. So to list the contents of a directory, the command ‘ls’ is used. To change the current directory, the command ‘cd’ was used. To count the number of words, characters or lines, the command ‘wc’ was used. All these commands were meant for a specific purpose. But what if I want to get another work done, that is not possible with a particular command, should i program a new command. What if I am not a programmer, it will be really difficult to create a new command. So there must exist a simpler option. And here comes the idea of piping. As we saw earlier, pipe is used to transfer an object or simply it gives out what is fed into it. So what if the output of one command is fed as an input to the other command. So we need a mechanism to transfer the contents of one command as an input to the other command. And for this purpose, piping is used in many Linux (Unix) based variants.

So a pipe transfers the output of one command as an input to the other command.

Take for example, I want to know the number of entries in a directory. As we saw earlier, ‘ls’ is used to see the contents of a directory and ‘wc’ is used to count the number of lines. What if we combine these two commands using a pipe, we get our work done. We feed the output of ls as an input to wc using a pipe.

In Linux, the symbol | is used for the purpose of piping

ls | wc -l

The option -l is used to get only the number of lines displayed. Thus we are able to get the number of entries in a directory without creating a new command. The best way to reuse the existing resources to get things done.

The idea of piping helped in a way that the users need not learn a large number of commands, but a very few number of commands. With this limited number of commands, one can get all the things done required to get their daily jobs done.

Compare this to the present day state of softwares and applications. A large number of softwares doing the same thing but with different user interfaces. What’s the result? A regular user is baffled to make a choice of a software ending up reading multiple review sites to make a choice. Sometimes, it is good to look at what people did in the past when they had a crunch of resources. Those lessons are helpful in making good design decisions.

Data Processing

One way to look at the Computer is as “A machine that performs data processing.” When you look at the computer from this point of view, you need to look at every device associated with the computer from this perspective. Before delving deep into the topic of Data processing, let’s first understand what is meant by the term ‘Data Processing’. The word processing means ‘A series of actions, changes, or functions bringing about a result’. Data processing means that you act upon the data to turn into other meaningful data. So there is something before and after the act of processing. That something before processing is often referred as input. The something after processing is called output. In the case of data processing, both input and output is the data. The input and output data may or may not be similar. If they are similar, the data may not have been acted upon. An intuitive example is when you use a clean tube to transfer water. Water(input) is passed through a tube from one end of the tube and you get the (same)water at the other end(output). But in most cases, we use the computer or any device to get a different output from the input we fed in. In case of computer, we often call the output data as (meaningful) information.

Let’s take some real word examples to see what processing means. Take a juicer. What you feed in the juicer are the peeled fruits and you get as outputĀ  pulp extracted from the juice. You can also see a change in state. The input was in solid state and the output is in liquid state. Thus a juicer is also a processing machine. It processed upon the fruits and returned the juice of the fruits as output.

A computer is a data processing machine. A computer has many devices attached to it. When you press a key on your keyboard, it generates electric signals which must be transformed to the pixels on the screen. The computer aids in this transformation to display the key at the right place on the screen. So a set of electric signals (input) is transformed to another set of electric signals(output) to display the pixels in the desired color. We abstracted out many inner details of this processing and presented an overall picture. What happens exactly is there are a number of data processors (any entity capable of data processing) working in a series (or in parallel) that is aiding the transformation. Each data processor does its intended duty to transform the data to another form of data. When all these data processor are arranged in a proper manner, we are able to see the character displayed on the screen.

Take any other example from our daily lives, we see the (data) processing. A camera, a mobile phone, a vacuum cleaner, a dish washer, a washing machine. All of these take something as input and give another meaningful or helpful output. All these devices process the data fed into them. The output is in the form that you want.