Understanding Computer Memory

In modern computer systems it is memory which is the main bottleneck. Hence understanding how memory works is very important to a programmer.

• Memory Basics is a excellent presentation from corsair memory.


• What every programmer should know about memory is a must read for developers.


• Memory Mountain technique is one very good way to characterize memory system performance

The Spectrum of Considerations involved in Software Engineering

The process of Engineering Software is an activity very different from most other types of engineering. Unlike other established fields of engineering you do not have absolute principles and a standard process enforcing them to deliver a product. All you have are experience, guidelines, best practices and coding techniques which can be followed in building software (Note that you do have the theory of computation involving Automata, Language theory etc. but their realization are mostly local to parts of a system and not to the whole. Thus witness the existence of lots of software developers without knowledge of the above-mentioned theories but able to successfully implement features of software systems). This is because of the complexity involved in building software. Where does this complexity come from? It comes from us trying to model the complex real/conceptual world through layer upon layer of abstractions to be executed on a finite resource machine in finite time. During the process of development we have to traverse back and forth through these different layers of abstractions keeping one or more aspects and features of the required end system in mind. Trying to juggle all these parameters simultaneously is tremendously taxing on our minds (see for example The Humble Programmer). One way to simplify the problem is to separate out the incidental activities involved in programming the system from the essential problem being solved. That way the commonalities involved in the art of programming can be applied to problems across various domains (for example i have developed software systems for the color matching domain and the computer network domain among others). Indeed this is how the profession of "Software Programming" has evolved i.e. by concentrating only on the activity of programming. But programming itself is just one consideration in developing a system. What are the others? How can we systematize our approach to these? How do they relate to each other? What aspects do we need to concentrate on to deliver an elegant and efficient system? In the rest of this article we will endeavor to answer these and related questions.

There are two important viewpoints to doing software development that we need to keep in mind;

• Top Down View : A top down approach is absolutely essential to zoom in and focus on your goals. What does the system do? This is the problem domain. You need to break through everything inessential right to the heart of the requirements. The system boundaries can only be defined using this approach.
  
• Bottom Up View : All systems exist in a certain context. A bottom up approach is essential to understand this context and its limitations. This context is defined by computer architecture, programming languages, runtime systems etc. and forms the solution domain.
  

Following the above viewpoints we look at the system as follows;

• Problem Domain : This consists of the concepts and their inter-relationships which we are trying to model in a computer system. The key is to build a conceptual model of what the user wants. The terminology used should be that of the user. This is often referred to as "Domain Engineering" or "Requirements Analysis". The resulting conceptual model should be documented and communicated to the user (using one or more modeling tools/languages like UML etc.) Interaction with a domain expert who may or may not be the actual end user greatly aids in this activity. For example when i had to develop a software package for the Color Chemistry domain, i had access to a professor and a industrial expert who greatly helped me do "Domain Engineering".
  

--To be continued

The myth of the "Expert" and how to become one

There is something very fundamentally wrong with our absolute belief and trust in so-called "Experts" and "Pundits" and their utterances. Now don't get me wrong; experts are needed. For example; since i don't know much about "nuclear physics", i would need to consult with a person who has studied it and hence is an expert relative to me. What i have an issue with is the common man's absolute deference to them. It is pretty common in arguments for one-party to quote a "well-known expert" and claim victory irrespective of whether there is any merit to it. What has happened here is that one has given control to another person and has suspended his own intellect. This is how people get manipulated and have their free will shaped by others. Now i am not saying that you disregard experts completely; they do have their uses (they are a data bank in their area of specialization). What i am saying is that do not swallow wholesale what an expert says without applying your own independent thought and judgement. If after deliberation you are convinced, then accept it but with the caveat that you might change your mind on that later (as you gain more knowledge). This is discriminating judgement.

Now why do i feel this way? Because our current system (educational and social) is completely broken when it comes to anointing experts. All it takes to be labeled an "expert" is a certain period of study (eg: 15 years of education, a master's and then a ph.d) and/or a long period of experience (eg: 10+ years experience). Did the period of study actually result in assimilation of knowledge and has it given rise to new insights? If not you are not an expert, you are simply regurgitating stuff. Does the 10 years of experience constitute actual worthy experience or 1 year's experience repeated 10 times? In the former case you have grown intellectually and have something to offer whereas in the latter case you have stagnated and cannot be named an expert. It is especially important in the current times with the explosion of all sorts of data and news that we learn how to identify true experts whose word we can accept and whom we can ignore.

The essence of an expert is specialization. As somebody once said "An expert is one who knows more and more about less and less". In the course of gaining "expertise" one necessarily has to go deep into the subject of study but in the process if one loses the interrelationships between concepts in the same domain or across domains what has one gained? In focusing on a small piece one loses the organic unity of the whole i.e. "one misses the forest for its trees". But specialization is an inevitability since there is only limited time and unlimited knowledge. Thus the true way to becoming an expert is to hold both the details and the big picture simultaneously in mind while studying. This has to be practiced consciously until it becomes a habit. The second thing is to be clear about the difference between a concept and its formal symbolic representation. Symbols manipulate concepts to derive relationships and/or new concepts. If the source concepts are not well understood then symbolic manipulation becomes useless since one cannot infer anything from the results. This is especially important today with science being so advanced that a lot of concepts are conveyed mainly by symbolic means (i.e. using maths). Hence concentrate on understanding the concept and more importantly the abstraction behind it before manipulating its formal representation. Spend more time in understanding the concepts by consciously searching for different explanations and viewpoints from varied authors. The formalism will follow naturally and before one knows it one has become an "expert".

The Importance of exercising Imagination

As i think about how we can function at an optimal level, i am rather struck with the huge role that Imagination plays in it. I would like to reiterate what i had mentioned in an earlier post, i.e. "Imagination is the basis for creativity". Why? Stimuli from the external world is received through the five senses (sight, hearing touch, taste and smell) and then processed via the mediation of the mind. This the key concept i.e. the mind gives meaning to the stimuli (within one's frame of reference). Even hard-wired automatic responses can be controlled by the exercise of will-power, which is nothing but the mind in action.

Thus in order to do anything at all, the mind needs to be controlled. Any activity that we do is only possible because we are able to focus the mind on that one task (however short the time period might be). If we can extend the time for which we can concentrate on a subject of study then we can have a deeper understanding of it which in turn will lead us to different perspectives from which we can look at it and finally the ability to see it in relationship to other subjects. This is creativity.

Imagination can be defined as something (a concept, an idea) forming in your mind based on;

• Arrival of stimulus from one or more of the senses which then forms a basis for further inferences. The scientific method is based on this; you deduce from data.
• Creating something in your own mind which might not have a basis in physical reality as it can be perceived by our five senses. For example we cannot perceive the ideas of Quantum theory by our senses like we can the Newtonian laws of motion. It has to be imagined and then experiments devised to prove or disprove the imagined theory.

Because you are creating something in your mind; your thoughts (the stuff making up your mind) are solely occupied by it. This is the definition of "focusing of attention" leading to concentration. Thus the more imagination you exercise the more vividly (by involving emotion) the more concentrated the mind becomes.

So how can you exercise your imagination more fruitfully?

• Initially start with "something" you like to do without any external motivation. This is important; you will only spend time if you have intrinsic motivation. For example, i like reading detective, adventure and western novels.
• When you are doing that "something", try and shutoff your other senses from receiving any external stimuli which distract the mind. Thus in the above example while reading, i don't listen to music etc.
• Imagine a detailed and visual imagery regarding that "something" with as much emotional content as possible. By this process you are forcing the mind to be actively involved in the subject. This is the key concept; imagination is the result of the active participation of the mind in creating something. Thus in the above example i would imagine the main protagonists in the storyline, sympathize with the hero, imagine the context of the scenes etc.
• Now bring in as much of your five senses into play in your imagination of the above "something". The mind then becomes completely focused by itself since the five senses have all turned inwards.
  
• Try and maintain the state for as long as possible and repeat the process as often as possible.
  
• Next take up a subject for which you have only external motivation and go through the above process consciously. It will be harder but once you start on the imagination path you will become intrinsically motivated towards the subject. You will find the whole experience immensely satisfying and rewarding.

Creativity by intuitively understanding and assimilating ideas

Our educational system gives us a number of ways (oral instructions, written word, audio, video etc.) of getting information but no technique at all on how to assimilate that information and turn it into knowledge/wisdom. What i mean here is how do i get to the "aha" feeling which happens only when we really understand something intuitively? The feeling is analogous to getting the punchline to a joke. Try to recall and feel the emotional content at that point. There is a revelation and clarity which induces a very pleasant feeling of well-being. Some call it the flow state. It is this state of feeling which makes an idea persist in our mind with full comprehension. Once we have a whole bunch of ideas deeply assimilated then we can see inter-relationships between them thus giving rise to original creative thought.

This is something that i had been thinking over and studying for a while. I felt that we don't really assimilate much knowledge in the current educational system. You went to school and listened to your teachers because you had to. The focus was and still is, on getting through with good grades and getting a job. Something has to set off the spark in you towards real comprehension. While the stimulus maybe external and momentary (like that one teacher who made the subject interesting) the follow-through and continuation has to be from within i.e. self-motivating. As the saying from martial-arts goes;

"to show one the right direction and the right path, oral instructions from a master are
necessary but mastery of the art comes only from one's own incessant self-cultivation".

If you look at geniuses/scientists/philosophers they all have a unique way of looking at things. They come up with original creative thought by a deep understanding of concepts and the ability to see inter-relationships among them which others have missed. What techniques/modes of thinking can we employ to attain such understanding ?

• Have a full and deep understanding of concepts at an intuitive level before you represent/manipulate them using symbolic logic (i.e. mathematics). Symbols without an understanding of what they represent at a conceptual level create more confusion than clarification. This is the heart of abstraction.
  
• Look at a concept from different perspectives and through each of them imagine what other concepts they maybe related to. No concept exists in isolation. They always exist in a network of related concepts.
  
• So what is a concept? It is an abstraction of a class of "things" (the class can be a set of n or 1 "thing"). The "thing" itself can be material or purely made up in mind. As long as it can have a distinct identity (i.e. can be separately held in mind from others) it is a valid "thing".
• Eg 1: A single apple fruit is a thing. However we do not usually care about one single apple but deal with the whole class of apples. We identify certain properties of all apples like color, texture, taste, etc. and abstract it out as the concept "apple".
• Eg 2: As another example i can imagine the concept of "gravitational force". I cannot see or feel it but can think of it as distinct from any other force and can calculate/measure it.
  
• How do we define/model/understand a concept?
  
• Think visually. Imagination is the single greatest tool we have. Pictorially imagining something is key to understanding it. This needs to be practiced regularly. By imagining it try to feel the concept as a sensation. This feeling is what cements retention in the mind.
• Use analogies to define a concept. The mind has difficulty with abstractions and concepts for which there are no physical world things. But by using analogy to real world things the concept becomes more concrete in our mind and thus starts taking a distinct identity of its own and becoming clearer. Also since concepts always exist in a network of other concepts inter-relationships between them become visible too.
• Use some symbolic notation (eg: mathematics, pictures, graphs etc.) to represent concepts and manipulate them. This concretizes the idea further and helps communicating it to others.

The above, in my opinion, IS the secret to creativity.