As discussed here in the comments, one of the advantages of comparative data on placement is that they help fill in gaps left over by the PGR. That is, the PGR aims to measure the collective reputation of a department’s faculty, but faculty reputation does not necessarily predict the likelihood of placement by that department, perhaps because it does not necessarily predict the overall quality of education in that department nor the quality of preparation for the job market by that department. Comparative data on placement has the potential to provide insight on these factors. To illustrate this, I below bracket the top 50 departments by tenure-track placement rate** (Note: I removed three universities from the top 50 that reported fewer than 2 graduates per year, since small numbers may yield misleading placement rates), providing for comparison these department’s ranks from the 2011 “Ranking Of Top 50 Faculties In The English-Speaking World” by the Philosophical Gourmet Report. Please note that placement brackets are provided only to demonstrate the potential utility of these data. Since the data set is not yet complete, I do not recommend viewing these as authoritative brackets. Update: Please see this post for an idea of how I envision this project developing. I have released the spreadsheet containing the raw data and methods I have been using to compute these results, and welcome any/all corrections. As a reminder, I do not have data on the yearly graduates from many departments, listed below. (Those departments are welcome to send me their data, if available.)
Update 7/1/2014: It has come to my attention that Brian Leiter has aired some criticisms of this post on his blog and has publicly suggested that it (this post, not his blog) be taken down. I respond to these criticisms below.
Updates 7/2/2014:
- I changed some wording above from “ranking” to “brackets” and added a link to the spreadsheet. I have also changed the numbers in the below ranking to a grouping by bracket (where departments are listed in alphabetical order within brackets). This was a suggestion of Ned Block’s. We have been corresponding on statistical significance and I decided that his suggestion would help avoid making small differences between placement rates appear more important than they are. I have left in the PGR rank for comparison, although the difference in rank has been omitted for the reasons provided above.
- I have also added updates to my responses to Brian, based on some new statistical tests.
- I am adding a link to a chart that will help readers to visualize the total number of reported tenure-track placements and estimated graduates from each department, rather than just percentage of tenure-track placements.
Update 7/6/2014: I ran a completeness test for 5 departments selected at random using a random number generator. The tenure-track numbers for these 5 departments appears to be accurate. More below.
Departments for which I have data on yearly graduates, with those that are not in the top 50 of the worldwide PGR bolded and those with incomplete graduation data starred (*), listed by tenure-track placement rate/rank and group:
| Tenure-Track Placement Rate | Department | 2011 PGR Rank (Worldwide) |
|
|
|
|
| ≥90% | Princeton University | 4 |
| University of California, Berkeley | 12 | |
| University of California, Los Angeles | 15 | |
| University of Pittsburgh (HPS) | 6 | |
|
|
|
|
| ≥80% | Columbia University | 12 |
| Harvard University | 6 | |
| Rutgers University | 3 | |
| University of Tennessee | ||
| Yale University* | 8 | |
|
|
|
|
| ≥70% | Massachusetts Institute of Technology | 8 |
| Stanford University | 10 | |
| University of North Carolina, Chapel Hill | 10 | |
| University of Pennsylvania | 34 | |
|
|
|
|
| ≥60% | Baylor University | |
| Fordham University | ||
| New York University | 1 | |
| Northwestern University | 38 | |
| Saint Louis University | ||
| University of California, Irvine (LPS) | 34 | |
| University of Chicago* | 24 | |
| University of Michigan, Ann Arbor | 5 | |
| University of Pittsburgh | 6 | |
| University of Wisconsin, Madison | 27 | |
| Vanderbilt University | ||
|
|
|
|
| ≥50% | Australian National University | 15 |
| Boston University | ||
| University of California, San Diego | 27 | |
| University of Connecticut, Storrs | ||
| University of Notre Dame | 21 | |
| University of Oregon | ||
| University of Texas, Austin | 24 | |
|
|
|
|
| ≥40% | Cornell University | 15 |
| Duke University | 29 | |
| Indiana University, Bloomington | 29 | |
| Johns Hopkins University | 49 | |
| Syracuse University | 49 | |
| University of Southern California | 12 | |
| University of Toronto | 15 | |
| University of Virginia | 49 | |
| University of Washington | ||
|
|
|
|
| ≥30% | Brown University | 22 |
| Carnegie Mellon University | ||
| Georgetown University | 45 | |
| Rice University | ||
| University of California, Davis | ||
| University of California, Riverside | 38 | |
| University of Colorado, Boulder | 29 | |
| University of Iowa | ||
| University of Maryland | 38 | |
| University of Sheffield | 45 | |
|
|
|
|
Here is the link to the chart mentioned above (last updated 7/2014). The blue bars represent the number of candidates with reported tenure-track placements between 2011 and 2013, whereas the red bars represent the estimated total number of graduates in those years, listed in order of tenure-track placement rate.
Departments for which I do not have data on yearly graduates, with those in the top 50 of the PGR bolded, listed by 1) tenure-track placements, 2) total placed candidates, 3) alphabetical order: University of Oxford; University of St Andrews/Stirling; University of Cambridge; University of Western Ontario; University of Chicago (CHSS); University of Kentucky; University of Edinburgh; University of Sydney; University of Warwick; York University; King’s College, London; London School of Economics; Marquette University; State University of New York, Buffalo; University of Adelaide; University of Auckland; University of Leeds; University of Melbourne; University of New Mexico; Victoria University of Wellington; Birkbeck, University of London; Durham University; McGill University; Trinity College, Dublin; University College London; University of Tasmania; Wayne State University.
**The average yearly tenure-track placements reported at ProPhilosophy and PhilAppointments between 2011 and 2014 divided by the average yearly graduates between 2009 and 2013 as reported in the 2013 APA Guide to Graduate Programs or by email–see this post for details.
Responses to the criticisms contained in Brian Leiter’s post, found here:
There are many things I would object to in Brian’s post, especially phrasing that calls into question my intelligence (“I would think philosophers are smart enough…”), but I want to focus on those points I take to be the most constructive:
1) I agree that the information is incomplete, but I think it is ok to air it in the fashion that I do for several reasons: a) the departments listed, at least in the United States, are likely to be in the same position with respect to missing data, b) the departments listed have reason to make their tenure-track placements public and I have used the most complete databases containing this information for the years in question, updating that information when called upon to do so, c) this exercise brings to light several departments that have been overlooked by the PGR’s ranking system, which I take to be valuable information to potential graduates, and d) my bolded caveat makes it clear that this is an exercise with the aim of making public the value of this sort of information, rather than a claim to an authoritative ranking. I have put out several calls for more data, including two posts at the beginning of June (here is the complete list of my posts). In 2011-2012 and 2012-2013 ProPhilosophy emailed departments directly to obtain complete placement information. If anyone knows of placements that have not been reported in the listed venues, please do send that information on to me.
Update (7/2/14): I have been getting new placement data by email and have started by updating the rankings. I will format the additions as the next step. Thus, if you notice some unformatted text at the bottom of the “Updated Data” tab, that part is a work in progress.
Update (7/6/2014): I ran a completeness test for 5 departments. I first generated 5 random numbers between 2 and 92 at http://www.random.org, corresponding to the row numbers in the far-right table of “Department Trends” tab. I then used the placement pages for the corresponding departments to check my numbers.
a) “56,” Washington University, St Louis. Missing from Excel spreadsheet: 3 postdocs (2012, 2013, 2014) and 1 renewable instructor (2013). TT placements from Excel: 4. Graduation years for tenure-track placements: 2010, 2011, 2011, 2012. Result: no change in the number of TT positions.
b) “34,” Syracuse University. Missing from Excel spreadsheet: 2 postdocs (2012, 2013), 1 VAP (2012), and 1 lecturer (2012). (Note: Airforce Academy appears to be 100% adjunct.) TT placements from Excel: 5. Graduation years for tenure-track placements: 2005, 2007, 2008, 2013, 2013. Result: no change in the number of TT positions.
c) “11,” Massachusetts Institute of Technology. Missing from Excel spreadsheet: 1 postdoc (2013). TT placements from Excel: 13. Graduation years for tenure-track placements: 2009, 2010, 2010, 2011, 2011, 2011, 2011, 2012, 2012, 2012, 2012, 2012, 2014. Result: no change in the number of TT positions.
d) “77,” University of Rochester. Missing from Excel spreadsheet: None/Unclear. (Note: Hobart and William Smith and Malone College placements appear to have occurred in 2005, according to university websites.) TT placements from Excel: 1. Graduation year for tenure-track placement: 2012. Result: no change in the number of TT positions.
e) “82,” State University of New York, Binghamton. Missing from Excel spreadsheet: 1 TT placement (2012). TT placements from Excel: 1. Graduation years for tenure-track placements: 2012, 2012. Note: Excel TT placement from PIC program, so TT placement number stays the same. Result: no change in the number of TT positions.
Thus, it seems reasonable to suppose that the data I have is fairly complete with respect to tenure-track data, but fairly incomplete with respect to temporary positions, such as postdoctoral, VAP, and instructor positions. (I will add the missing data yielded by this exercise soon.)
2) Choosing the relevant PGR ranking: I chose the most recent PGR ranking because of my assumption that faculty reputation makes more of an impact on how one is perceived on the job market than it is a measure of the quality of one’s education. One could look at the mean PGR ranking over the course of each graduate’s education, but I suspect that the impact of this value on placement rate is far outweighed by the PGR ranking/faculty reputation at the time of that graduate’s placement. I have heard anecdotal stories of this sort, in any case. I am happy to hear more argument on this point and could possibly run an analysis that would test the point.
Update (7/2/14): on the suggestion of Shen-yi Liao I looked at the correlation between the tenure-track placement rates for these departments and both the 2006 and 2011 PGR mean ratings for these departments. The correlation coefficients for both years are 0.7. I thus see no reason to use the 2006 numbers here, since the 2011 numbers appear to be equally correlative.
3) I treat tenure-track placements equally here because I think this is the most useful information to potential graduate students. I presume that most potential graduate students want a tenure-track equivalent job but do not yet know what type of job. I know for a fact that some graduates seek positions with low research expectations and high teaching loads, whereas others seek positions with high research expectations and low teaching loads. I also know that location is more important to many graduates than the type of institution and/or department. Finally, I know that climate matters a lot to many graduates seeking placement. Perhaps in time we can look at more fine-grained measures that will be useful to those potential graduate students who have a better idea of what they want.
4) I am having a difficult time reading much of this point charitably in its literal form. Obviously, it would be very surprising if the numbers that I have collected on behalf of these departments are “equivalent, in most cases” to randomly chosen numbers. I compare the average yearly graduates to the average tenure-track placements to arrive at placement rate. For the most part, more years of data will yield a more accurate placement rate, so long as one is as consistent as possible across departments. In my case, I was able to obtain 5 years of graduation data for most departments through the APA Graduate Guide for 2009 to 2013 and had 3 years of placement data for many departments through the collection efforts of ProPhilosophy, who emailed departments directly, and PhilAppointments from 2011 to 2014. Since I compared average to average and the years overlap for the most part (that is, the graduates earning placement are mostly graduates between the years 2009 and 2013), I don’t find this issue very problematic. I am open to more discussion on this, of course.
Update (7/2/14): on the suggestion of Ned Block, I looked at the statistical significance of the difference in placement rate between 3 departments. Brian may have been speaking to a supposed absence of statistical significance when he claimed to see my numbers as equivalent to random numbers. Since Ned is at NYU and since NYU is put forward as a counterexample by Brian, I looked at the significance of the difference between NYU and both UC Berkeley and UC Riverside (since these are the only two departments with the same average yearly graduates to NYU). Using the chi-squared test I found the difference between all three of these to be highly significant (p=.003), and so this difference is not likely due to randomness. Further, I looked at only UC Berkeley and NYU and found that difference to be statistically significant (p=.03). I hope that this puts the worry about randomness to rest.
Update 7/6/14: from the exercise listed under 1, the average graduation year for candidates placed in tenure-track jobs between 2011 and 2014 from these 5 departments is 2011 and only 4 of the 25 placed candidates (16%) obtained their PhD outside the timeframe of the graduation data that I used (2009-2013). This is further reason to suppose that the graduation data I used captures most of the placed candidates (84% of this set).
I am happy to further respond to constructive criticisms on this report. I would ask that these criticisms focus on the work, rather than on my person.
New APPS 
Leave a comment