This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.5 India License.
Caste System - One of the fundamental constants of Indic socio-polity.
Caste is one of those aspects of Indic Socio-Polity which is intermingled closely with all the other concepts of Dharma-Artha-Moksha. I have left out Kaama, because the link between caste and "desire" is tenuous. Caste is one of those "Common Factors" of India which somehow has persisted and been retained for good OR evil. Whilst everyone knows what political measures are required to eradicate the system, no one is willing to implement them which tells a lot about the "necessity" of this system to the ruling class of India (whatever caste they may be of).
Megasthenes mentions 7 different "categories" of people found in Indian society while his stay in India. 800 years later, Chinese traveller Fa'Xien too notes "castes" and mentions about "Chandalas" which was a category of outcasts with whom any formal societal interaction was prohibited in times of Vikramaditya. There is plenty of material which is available for the readers to investigate the "Origins" of this socio-political system in India. The most popular and commonly accepted theory is that Caste system is derived from "Varna-System" of ancient India which divided people into intellectuals, warriors, traders/artisans/farmers, labourers (Brahman, Kshatriya, Vaishya, Shudra respectively).
While there is close relation of Varna and Birth (The concept of Dwija-Twice born), the term Jaati came to be used as substitute of "Varna" with time. How did it happen, it is difficult to tell. But popularisation of the word "Jaati" began the conversion of "Guna-Karma based" Varna system (as described by Vyasa in MBH) to birth-based caste system. This may be true vice-versa as well, although in my personal opinion first possibility is more probable. While there are few stringent strictures about "Shudras" in Manusmriti, how much Manusmriti was popular in Pre-British India is matter of speculation. There were other smritis like Medhatithi Smriti, Yagnavalkya Smriti, Naarad Smriti, Brahaspatismriti which dealt with Indic jurisprudence. All of them however are unanimous about the "twice-born status" of Brahmin-Kshatriya-Vaishya Varnas. Regarding the question "What is the precedence of commanding "respect" and "importance" that must be given to people, most of the dharmashastras and Mahabharat are unanimous about "knowledge, conduct, age, family, property" in this particular order. (ref. 3.17, Mahabharat; Yaagnavalkya Smriti 116)
The Ossification of birth-based caste system
The process when Indic religions started gelling together as one "hinduism" religion, marks the beginning of exploitative caste system, IMHO. This began after successful incursions of Ghuris in the heartland of gangetic plains and it spread southwards with khiljis and tughlaqs. Manusmriti is one of the thousands of other law books which were enforced in various parts of India in various times. One of the famous "Smriti" is "Medhatithi Smriti" enforced in Rajasthan, Gujarat, Harnaya, Punjab and eastern Sindh during the times of arab invasions and battle for rajasthan in 800's. Prior to Ghurid invasions, there was no need for a "hindu" identity. for social and political identification reasons, caste/profession/varna was enough, for spiritual identification, moksha-maarga was enough. The crystallization of "Caste" as a system in which the lowest strata was inhumanly exploited began at "mass scale" after the need for finding a common identity for a "non-Muslim" Indian began to arise acutely. This is where the "Mess" begins.
The only thing which is constant in this entire "mess" of caste system is "Convenience". Convenience is the hallmark of a Hindu. When it is convenient to be "a custodian" of Indian culture, Hindus act so. When traditions start becoming liability and become inconvenient, they are overthrown. This "convenience" of various groups of people in given region in given space-time is the constant factor of the complex web of interdependent caste system. This "convenience" is lot to do with the "mercantile and mercenary character" which slowly got inculcated into the psyche of Indic power-holders and public alike. This entire mess of reservation is also the same. the groups which fight for a "low class" status officially try and marry off their daughters in the castes which considered as high caste by them in their space and time. They also call themselves (strive to call themselves) as high born. Its all complex. Something which is convenient for particular castes in particular regions who wield power (of strength OR of numbers OR of money OR of information/administration) in given times, try to gain privilege and political space for themselves. Whatever means are required to elevate themselves are used.
In medieval times, the typical means was attestation of brahmin caste, hence all the "Daan-Dharma" and donations. In modern times, castigating brahmins and their "exploitations of 5000 years" in order to gain "recognition" as "backward castes" is necessary for official recognition. hence the case. In this entire process that social group with least resources to "lobby" itself in given space and time to get the share of the moolah becomes "Dalit". They either have to flee the region or suffer the weight of "advantageous groups" in their space and time. When times change, the classes move up and down and every time, one at the bottom suffers. All the groups hence try to move up at all costs, justifying the story of Birbal and monkey who tries to save her life by standing on her kids. There is a reason why Islamic raids are mentioned every time lignification of caste system is discussed. As I said earlier, the one on the lowest strata bears the weight of everyone above, this also opens them as a "market niche" for other competing systems. Before Islam, Dharma (Duty/profession/parameters of judging a deed as righteous or not) was constant for a particular caste/varna, what changed was preferences for moksha (if any).
In earlier days of Islam in India (prior to Ziauddin Barrani and his Islamic Caste system), this market niche was open for conversion to Islam. The converts within higher varnas and castes (by higher, I mean advantageous in given space-time) were earned by force and lucrative coercion (tax benefits/promotions/waiver in octroi/waiver from Jaziya) and this is well documented. The "lowest castes" converted to Islam just as they would have converted to Buddhism OR anything else (which they might have before anyways). The response of this newly forming "Hindu" society against this visibly foreign system was to become rigid and conservative. Sufism, Jaziya coercion and sword was the combined weapons used by earlier Islam to target various market niches of Indic society to win converts.
Here again there is an added variable. 99% of "janta" lived in villages. It is difficult to convert this entrenched janta. Primarily those who had to travel a lot (artisan classes, for example) were converted rapidly owing to disentrenched society and trade benefits. The response of Indic people towards these people was typically that of extreme "disgust". One of the reason for this disgust might be "beef consumption" by the neo-converts. Abhoration of beef is one of those inexplicable memes which seems to have popularised itself for no particular selective advantage OR rather selective disadvantage. Most of the people who were forced to consume beef by Muslims and later by Christians (St. Xavier and his inquisitions) were thrown out of caste by "Hindus" making the job easy for others.
While lower castes OR groups "trapped" in lowest strata saw an opportunity to bypass the struggle of upward social mobility by simply converting, many of them (especially in gangetic delta and Indo-gangetic plains) converted to Islam. This attempt was however undone by Ziauddin Barani who created a caste system is Islam as well. The famous Ajlaf-Ashraf divide begins the interesting phenomenon of "Islamic-caste". Same is with Christian conquistadors of western coast. This shows that caste is the only constant in India, language, moksha-maarga, dharma, religion, ethnicity everything is variable.
This is primarily because India is a vast geography with a natural barriers. This is an ecosystem in itself which relatively isolated from central asia and tibet. It is important for Indic political parties to periodically try and consolidate the Northern and north-western part of India politically in order to maintain their political space. For building that force required for consolidation, every part of society needs to be given a social, a political and a economic space and assurance. In many ways, caste coupled with "Joint Family system" was India's version of "Social-Security program". A temporarily OR permanently "non-productive member" was sort of "taken care of" by rest of the productive members. Furthermore, as Jawaharlal Nehru has argued, this system (in its pre-lignified form as well as to certain extent in later ossified structure) enabled staged assimilation of foreign tribes into Indic genepool.
To understand this aspect of caste system one can study the case of Mr. Ramalinga Raju, the infamous chairman of Satyam group. He called for meeting of his caste people and raised money to show the assets and ease away company's transfer. Why did they help him out, perhaps because when was a contributing member, he helped others (of his caste). This is the typical nature of a caste system. To strengthen this bond, marital alliances were made and to avoid "inbreeding" gotra system of Brahmins was popularised amongst all other castes (except for the lowest ones in given space-time). When lowest ones stopped being lowest and became highest (shephards like Holkar) they too adopted this system. This is where the question of "Shudras" and "Kshatriyas" come into picture.There were "Classical" kshatriyas to south of Narmada long before Marathas. Saatvhanas, Cholas, Hoysalas, all the way up to Yadavas of Devagiri were "classical Kshatriyas" and all of them were to south of Narmada.
It is more to do with "Social Mobility". A landless labourer is a Shudra. If he does something and wins small piece of land, he becomes Vaishya. With time, he (or his progeny) owns a large estate of land, supports lots of people many of who join his "personal army" to "win more land". Thus the same family becomes kshatriya. This transition is socially attested by two forces. The incumbent power-centre of the region (that is King) and Brahmin. Both King and Brahmin attest this rising "star" for their own reasons. King gets an additional shot of strength in arms and additional revenue. Brahmin gets donations. The "Rising star" gets recognition and hence prospects of further and fast rise. This is further cemented by marital alliances.
Example - The grandfather of Shivaji Maharaj shows similar pattern. Maloji Bhonsale (Shivaji's grandfather) was an ordinary bargir (horseman) and his father Babaji Bhonsale was a ordinary common peasant. Maloji was a brave soldier and died fighting for Lakhuji Jadhav (the classical Kshatriya and descendent of Yadavas of Devgiri). Lakhuji gave Shahaji (Maloji's son) his daughter in wedding. Thus Jijabai was wed to Shahaji. With time, the political career of Shahaji was rising rapidly, so was his property and his social standing. The need for attestation began to arise acutely.
Ghorpade clan was trying to show their lineage from Sisodiya Rajputs of Rajasthan in court of Adilshah. Shahaji proved that he had stake in the "property" of Ghorpades by showing some distant relation. Thus, Shahaji (and hence Bhonsale clan) was declared as Sisodiya Rajputs by Adilshah. With time Shahaji's son, Shivaji (whose calibre we all know) became a first Hindu king to be coronated in Vedic style after Rashtrakutas in Deccan. Thus he came avatara of Vishnu himself on earth. There were brahmins who were making noises (few Brahmin-clans from Nashik-Maharashtra) and there were other Brahmins who were supporting this move (Brahmins from Banaras, UP). Both had their own selfish interests. Everyone was satisfied by donation of one crore rupees and attestation was complete. Shivaji and his house reaped rich dividends of this attestation in coming 140 years. Owing to this recognition, they stood as natural successors to Mughals in North India. I will stop here.
Bhonsale's rose up in the hard way. The rise of Ibrahimkhan Gardi, OTOH, was simpler. He belonged a tribal Gaardi caste. Converted to Islam, showed talent in handling of artillery, was given title of khan by nizam, showed his talent at udgir, became artillery chieftain of marathas, died fighting for marathas at panipat and achieved eternal glory. The conversion eased his rise and social standing. This is not the golden rule, however. There are many examples (like Kalachand Ray, aka Kala Pahad) where conversion leads to desolation, tyranny and infamy.
All this is about post Barani society of India during Islamic times and how various castes interplayed with each other to give maximas and minimas of Indic civilization. Hence geographical distinction of castes is incorrect. Caste/Varna was already standardized all over India by then.