The City College of New York was originally founded as the Free Academy of the City of New York in 1847 by Townsend Harris. A combination prep school and college, it would provide children of immigrants and the poor access to free higher education based on academic merit alone.

It was subsequently renamed the College of the City of New York, but that name was later transferred to the complex of the municipally-owned colleges in New York City, which was the predecessor of the modern City University of New York. At that time, CCNY became officially City College of the College of the City of New York, and later adopted its current name when CUNY was formally established as the umbrella institution for New York City's municipal-college system in 1961. The name City College of New York, however, is in general use.

In 1847, New York State Governor John Young had given permission to the Board of Education to found The Free Academy, which was ratified in a statewide referendum. Founder Townsend Harris proclaimed,

"Open the doors to all ... Let the children of the rich and the poor take their seats together and know of no distinction save that of industry, good conduct and intellect."

Dr. Horace Webster , first president of The Free Academy on the occasion of its formal opening, January 21, 1849, said:

"The experiment is to be tried, whether the children of the people, the children of the whole people, can be educated; and whether an institution of the highest grade, can be successfully controlled by the popular will, not by the privileged few."

In 1851, a curriculum was adopted which had nine main fields: Math, History, Language, Literature, Drawing, Natural Philosophy, Experimental Philosophy, Law and Political Economy.

The Academy's first graduation took place in 1853 in Niblo's Garden Theatre, a large theater and opera house on Broadway, near Houston Street at the corner of Broadway and Prince Street.

In 1866, the name was changed to The College of the City of New York and lavender was chosen as its color, while in the next year, the academic senate, the first student government in the nation, was formed.

Having struggled over the issue for ten years, in 1895 the New York State legislature voted to let the college build a new campus. A four-square block site was chosen, located in Manhattanville, within the area which is today enclosed by the North Campus Arches.

Education courses were offered in 1897 as a result of a city law which prohibited hiring teachers who lacked proper education. The School of Education was established in 1921.

The College newspaper, The Campus , published its first issue in 1907, and the first degree-granting evening session in the United States was started.

Separate Schools of Business and Civic Administration and of Technology (Engineering) were established in 1919. Students were also required to sign a loyalty oath.

In 1947, the college celebrated its centennial year, awarding honorary degrees to Bernard Baruch (class of 1889) and Robert F. Wagner (class of 1898). A 100 year time capsule was buried in North Campus.

In the years when top-flight private schools were restricted to the children of the Protestant Establishment, thousands of brilliant individuals (especially Jewish students) attended City College because they had no other option. CCNY's academic excellence and status as a working-class school earned it the titles " Harvard of the Proletariat" and the "poor man's Harvard."

Even today, after three decades of controversy over its academic standards, no other public college has produced as many Nobe laureates who have studied and graduated with a degree from a particular public college. CCNY's official quote on this is "Nine Nobel laureates claim CCNY as their Alma Mater, the most from any public college in the United States". This should not be confused with Nobel laureates that earned the distinction at a public university as UC Berkeley boasts 19.

In its heyday of the 1930s through the 1950s, CCNY became known for its political radicalism . It was said that CCNY was the place for arguments between Trotskyites and Stalinists. Alumni who were at City College in the mid-20th century said that City College in those days made Berkeley in the 1960s look like a school of conformity.

CCNY is the only team in men's college basketball history to win both the NIT and the NCAA Tournament in the same year, 1950. However, this accomplishment has been overshadowed by a point shaving scandal in which, during the course of 1951, seven CCNY basketball players were arrested for taking money in order to shave points. This led to the decline of CCNY from a national powerhouse in Division I basketball to a member of Division III and damaged the national profile of college basketball in general.

During a 1969 campus takeover, under threat of a race riot, African American and Puerto Rican activists and their white allies demanded, among other policy changes, that City College implement an aggressive affirmative action program (Traub). At some point, campus protesters began referring to CCNY as "Harlem University." The administration of CCNY at first balked at the demands, but instead, came up with an open admissions or open-access program under which any graduate of a New York City high school might be able to matriculate either at City College or somewhere in the CUNY college system. Beginning in 1970, the program opened doors to college to many who would not otherwise have been able to attend college, but came at the cost of City College's academic standing (This view is mirrored in mass media of the day, however as a graduate from that period, who asked many long tenured faculty if in fact the curriculum had been changed, the universal opinion was that nothing had changed. The fact is that there were preparatory / remediation classes that one could take be able to matriculate, but as many did, after the first term, those who could not pass classes simply dropped out.)... and New York City's fiscal health.

(The fiscal health of the City of New York was a cheap regressive way to impose tuition, the blame for this disaster can be brought to bear with Mayor Abraham Beame and Governor Mario Cuomo, who made trade-offs to appease the Ford administration. (see Daily News headline: FORD to City: DROP DEAD) Again the fact is that tuition, which was imposed in a gradual manner (I recall paying $24 my last semester!), dealt a mortal blow to the long standing idea that education is a social right. The reason why a huge increase to bail out New York City was impossible to abruptly implement was simply because the vast majority of students came from working class families that could not afford to send their offspring to more expensive institutions. Such a policy would have caused the closure of the third largest university system in the country, and would have brought havoc upon City Hall in terms of dealing with one of the most powerful university faculty unions of the time. Competition was and is so extreme at CUNY because of the very well paid and well educated faculty, all thanks to their union. This also in fact encouraged top notch professors who would never have put up with weakening a long standing curriculum of excellence. I also recall having to take a final exam in Shepard Hall on the top floor where the roof had gone so long without repair it was actually raining in some parts of the under-heated classroom. So, the argument that the imposition of tuition was some sort of wind fall or realistically aided in saving City Hall from its economic woes does not stand.)

City College began charging tuition in 1976, and by the 1990s (Regan/Thacher era doctrine) stopped accepting and working with students who didn't meet its formal entrance requirements.

CCNY's new Frederick Douglass Debate Society defeated Harvard and Yale at the "Super Bowl" of the American Parliamentary Debate Association in 1996. In 2003, the college's Model UN Team was awarded as an Outstanding Delegation, an honor that it would repeat for four years in a row.

The U.S. Postal Service issued a postcard commemorating CCNY's 150th Anniversary, featuring Shepard Hall, on Charter Day, May 7, 1997.

In October 2005, Dr. Andrew Grove , a 1960 graduate of the Engineering School in Chemical Engineering , and co-founder of Intel Corporation, donated $26,000,000 to the Engineering School, which has since been renamed the Grove School of Engineering. It is the largest donation ever given to the City College of New York.