The Conference of Southern Graduate Schools, founded in 1971, succeeded the Conference of Deans of Southern Graduate Schools. The Conference of Deans had been established in 1925 out of a committee meeting of the Southern Association of Colleges and Secondary Schools at Charleston, South Carolina. The members of this committee, Deans Bocock of the University of Georgia; Puryear of the Texas Agricultural and Mechanial College; and Metcalf of the University of Virginia, met in Washington in 1926 with the United States Bureau of Education. After this meeting they decided to call a meeting of deans in Southern universities to discuss common problems and policies.

The original purpose of the Conference of Deans was to work with the United States Bureau of Education compiling information on graduates of Southern colleges for admission to graduate schools. This function was soon superceded by other agencies, so the organization served as a focal point for discussions about regional problems in higher education. In 1960, a national organization, called the Council of Graduate Schools, was formed. By 1963 the Conference of Deans decided that a regional group, such as itself, no longer served any purpose and therefore dissolved.

In 1969 an informal discussion between some graduate deans in the South led to several ad hoc meetings. These laid the groundwork for the re-establishment of a Conference of Southern Deans which was renamed the Conference of Southern Graduate Schools. The ad hoc committees decided that the dissolution, in 1963, of the Conference of Deans had been a mistake; therefore, it was decided that in order for the South to have prosperous, and respected graduate school programs, a re-constituting of the Conference of Deans was needed. In May of 1972 the conference adopted a constitution and by-laws, thus re-establishing a single powerful voice, both internally and externally, for the South's graduate education. The leadership behind these efforts came from Deans Leonard Greenfield, University of Miami; William Toland, Baylor University; and John Harrison of the Univeristy of Miami.

The purpose of this organization is primarily the consideration of mutual problems among the member institutions relating to graduate study and research. The Conference continues to function with almost one hundred active members and an annual conference each February. Various committees actively gather information and strengthen the work of graduate schools in the South through this well-respected, and highly regarded organization.