A few days ago I posted a list of features that I take to be essential to an ideal report on placement, seeking comments and suggestions. One of the features I mention there is recency. All departments are likely to place more candidates given more time, but this slope is steeper for certain departments. Moreover,  placement varies year to year. Thus, one's choice of time frame can substantially alter data on placement. This is the reason that Brian Leiter's numbers for NYU look better than mine (here and here)–I looked at the years 2012 to 2014 (3 years in the recent past), whereas he looked at the years 2005 to 2010 (6 years in the distant past).* Looking at NYU's placement page, one can easily see that the percentage of graduates placed in tenure-track jobs drops as one reaches the present. As I said, this is likely true for all departments. This means that if you look at data in the distant past, it might not matter what the length of the time frame is, but if you look at data ending in the recent past, the length of time frame makes an impact. That is, for NYU for the years starting in 2005, a 6-year time frame has 87% TT placement, a 5-year time frame has 90% TT placement, a 4-year time frame has 88% TT placement, and a 3-year time frame has 90% TT placement. But for the years ending in 2013, a 6-year time frame has 69% TT placement, a 5-year time frame has 65% TT placement, a 4-year time frame has 56% TT placement, and a 3-year time frame has 56% TT placement. Note that even the 6-year window ending in 2013 is associated with much lower placement than any of the windows starting in 2005. It seems obvious to me that we should favor more recent data, since they reveal which departments place students more quickly than others and since they are more relevant to students looking at graduate programs. Beyond that, it is not obvious just what length of time we should choose (3, 4, 5, or 6 years) or just which year we should use as the endpoint. 

Yet, one's choice of time frame has a large impact on comparative placement data. Let's compare NYU's placement page to the placement pages of those departments that I found with these methods to have the highest tenure-track placement rates: BerkeleyPrincetonPittsburgh HPS, and UCLAIf we look at NYU's worst time frame it comes out behind all the others (2010-2013: NYU 56%, UCLA 59%, Berkeley 63%, Princeton 65%, and Pittsburgh HPS 88%). If we look at NYU's best time frame it comes out ahead of all the others (2006-2009: NYU 94%, UCLA 67%, Berkeley 78%, Princeton 86%, and Pittsburgh HPS 93%). If, on the other hand, we look at multiple time frames then a new type of comparison is possible. We can determine, for example, which department has the least low value for tenure-track placement, given any time frame in the period from 2005 to 2013 (with a 3-year minimum time frame and a 6-year maximum time frame). In that case, Pittsburgh HPS comes out on top. It's lowest value is 85%. In comparison, the lowest value for Princeton is 65% (2010-2013), the lowest value for Berkeley is 59% (2009-2012), the lowest value for UCLA is 52% (2009-2012), and the lowest value for NYU is 56% (2010-2013). So if we look at the least low placement for all of these time frames, NYU comes out second to last. Finally, if we look at the full range, from 2005 to 2013, NYU comes out in the middle (Pittsburgh HPS 93%, Princeton 76%, NYU 74%, Berkeley 70%, UCLA 65%). 

Suffice it to say, these decisions make a substantial impact on one's results. For that reason, one should attend carefully to justifications on recency and time frame. I will remove the links to Brian Leiter's two posts on placement data here, since I am concerned that they will mislead students. If I had written those posts, I would certainly take them down knowing what I have made clear in this post (i.e. that the numbers for NYU are inflated for the very time frame that Brian Leiter chose to look at, relative to other departments). I have emailed Brian a link to this post.

As for my data, I use the years 2012 to 2014 because those are the most recent years and the years for which I have large data sets. (ProPhilosophy was kind enough to email departments directly in 2012 and 2013, which substantially increased the number of reported hires for those two years.) To go prior to 2012 I would have to either look at individual placement pages for all 118 departments, many of which do not have data of the sort I need, or use what I know to be a skewed sample from the Leiter Reports blog. I have made clear that any rankings I produce are a work in progress and should not be taken as authoritative. (That is one reason I post them to blogs, and not an independent website.) But as time goes on and this process is improved I will have to start making decisions about which time frames matter. I may well follow the lead of David Marshall Miller in reporting multiple time frames, since this might be helpful for students. Suggestions on this point are welcome. (The data that I used for this post are after the break. Feel free to suggest corrections where needed.)

*I hope that this does not need saying, but I am not picking on NYU here. One of my dissertation advisors was at NYU and one of my best friends is currently a student there. I am looking at NYU because it appears to be a focal point in Brian Leiter's criticism of my work. If one were to look at other measures beyond just tenure-track placement, NYU may well fare better than it does here. 

Update (7/14/14): In order to satisfy the worry that NYU is particularly burdened by graduates of the JD/PhD program in this measure (2 graduates from NYU left academia for law in this time period, compared to 1 from Princeton, 3 from Berkeley, and perhaps 2 from UCLA), I compared NYU to these other programs while leaving out all those graduates who left academia. In that case, as I point out in the comment below, it is still clear that time frame matters and, in particular, that the time frame of 2005-2010 overly inflates NYU's record (2008-2013 puts NYU in the middle of the group, at 80%, whereas 2005-2010 puts it at 95%, square with Berkeley and Pittsburgh HPS, ahead of UCLA and Princeton. It might be worth noting that with the same methods Fordham University placed 69% of its graduates into tenure-track jobs between 2008 and 2013). See my comment below for details.

  NYU UCLA Berkeley Princeton Pittsburgh HPS
  TT Placements Graduates TT Placements Graduates TT Placements Graduates TT Placements Graduates TT Placements Graduates
2005 2 3 4 4 2 3 5 5 4 4
2006 4 4 2 4 2 3 8 8 8 8
2007 3 3 4 5 2 3 4 5 3 4
2008 5 6 3 3 5 5 9 11 0 0
2009 5 5 1 3 5 7 3 4 3 3
2010 1 2 3 4 2 2 2 4 1 1
2011 4 6 4 6 4 7 5 5 2 2
2012 4 6 3 8 4 7 7 11 2 2
2013 1 4 6 9 2 3 8 14 2 3

 

(6 years) NYU UCLA Berkeley Princeton Pittsburgh HPS
2005-2010 87% 74% 78% 84% 95%
2006-2011 85% 68% 74% 84% 94%
2007-2012 79% 62% 71% 75% 92%
2008-2013 69% 61% 71% 69% 91%
(5 years)          
2005-2009 90% 74% 76% 88% 95%
2006-2010 90% 68% 80% 81% 94%
2007-2011 82% 71% 75% 79% 90%
2008-2012 76% 58% 71% 74% 100%
2009-2013 65% 57% 65% 66% 91%
(4 years)          
2005-2008 88% 81% 79% 90% 94%
2006-2009 94% 67% 78% 86% 93%
2007-2010 88% 73% 82% 75% 88%
2008-2011 79% 69% 76% 79% 100%
2009-2012 74% 52% 65% 71% 100%
2010-2013 56% 59% 63% 65% 88%
(3 years)          
2005-2007 90% 77% 67% 94% 94%
2006-2008 92% 75% 82% 88% 92%
2007-2009 93% 73% 80% 80% 86%
2008-2010 85% 70% 86% 74% 100%
2009-2011 77% 62% 69% 77% 100%
2010-2012 64% 56% 63% 70% 100%
2011-2013 56% 57% 59% 67% 86%
Posted in ,

58 responses to “Why Recency and Time Frame Matter”

  1. Carolyn Dicey Jennings Avatar

    “Omitting such information” is not the same thing as just not gathering the information. As I say above, it is a lot of work to do so. Further, it requires more judgment calls. I have tried to avoid making too many of these (e.g. do I use the “Spec/Faith” Carnegie classification, do I try to assess the mission statement, or do I try to figure out which universities require statements of faith?). More judgment calls, more distinctions, more statistical tests all leave one more open to error, taking away from accuracy. One thing that is clear to me is that graduate students care about the distinction between tenure-track and temporary contracts. In so far as the data I have presented are accurate with respect to placement in the former, it should be helpful for students. I think that students likely care about many other distinctions, too. If I manage to bring those to bear on the data, the same point about accuracy and helpfulness will probably apply there. The very nature of this type of effort means that I cannot provide a full picture of what is happening at each department. And anyhow the point of these types of enterprises is to help students with what is otherwise an overwhelming amount of information. So some factors will necessarily be lost. Balancing accuracy, efficiency, clarity, and informativeness is a tough call. I think I could probably add more on the side of informativeness. But I do not concede that what I have done here is misleading on this front. It should be perfectly obvious that the data only capture tenure-track jobs, and not types of tenure-track jobs.

    Like

  2. anononymous Avatar
    anononymous

    Hi Carolyn, I agree with you that trying to make finer distinctions about quality of jobs, beyond appealing to things like the Carnegie classifications, is unwise. The vast majority of philosophers vote Democratic, and would prefer not to work in Red States. Many I know do not even apply for positions in those areas. Should those jobs be discounted? Most philosophers are uncomfortable with the sort of plutocracy that the Ivy Leagues are believed to sustain and perpetuate. Should those jobs be worth less? Many don’t want to live in “fly over” states, because they think they are not culturally rich enough. Etc. Etc. Thinking this way quickly becomes ridiculous, especially when so many false and irrational beliefs lie behind these judgments. Pretty soon the only jobs “worth having” would be in the UC system. 🙂

    Like

  3. Darius Jedburgh Avatar
    Darius Jedburgh

    Christy Mag Uidhir at 49: Catholic schools ‘actively condemn or marginalize’ drinking and smoking, which are ‘more or less institutionally ignored at secular schools’?
    This strange claim seems to express a conception of ‘secular’ culture as largely free from religiously-infected norms. This conception is very widespread but I think it is quite wrong.
    The infrastructure of US ‘secular’ culture is emphatically protestant. The mainstream of that culture, including its manifestations in ‘secular schools’, condemns and marginalizes drinking and smoking much more actively than any catholic institution qua catholic. This is just an example; the point is general. Another example would be the unmistakably white, protestant character of much of the current ethical thinking of academic feminists. (This is to point to a deficiency in the cultural-historical self-consciousness of that thinking; it is not to criticise that thinking itself. That would depend on what one thinks of the ethical inheritance of white North American protestantism.)

    Like

  4. Pastrami Avatar
    Pastrami

    I think that even among TT jobs it is highly relevant whether one is most likely to compete for research-oriented TT jobs (2/2 and less), teaching-oriented TT jobs (3/2-3/3), or research-killing TT jobs (<3/3). This makes a huge difference. It is true that a lot of people are happy to accept, say, a 3/3 or even 4/4 job, but this is usually not because they were shooting for it to begin with. I think most people come to the graduate school because they want to become serious contributors to the field and only once the gloomy job market is upon them (or once they see their work might not really position them in that way), adopt a more humble attitude. Let’s put it this way: how many PhD’s from Fordham are at departments comparable to the Fordham department itself, that is, at departments producing PhD’s for the market at which Fordham mostly competes or at departments/universities with similar standing? And what is the market for Fordham’s own hiring – places comparable to Fordham or places, say, higher in the PGR food of chain? Now take UCLA and ask the same questions. I do not suppose we would like to employ these sort of comparative acrobatics to achieve some rankings, but it seems to me that something like this, if anything, is of each students must figure out for herself from her own perspective. Ranking Fordham and UCLA as, say, equal in placement, without these kinds of comparisons does not seem to me particularly useful when it comes to choosing the department for its program’s quality. One can check whether people get jobs simpliciter pretty easily now.

    Like

  5. Christy Mag Uidhir Avatar

    Darius, at no point did I target any particular school let alone Catholic institutions as a whole. The overtly religious schools in to which Fordham placed its recent graduates, while mostly Catholic, are certainly not exclusively so but also include Seventh-Day Adventist, Methodist, and Non-Denominational Christian. I merely pointed out that atheists, as most philosophers are, might find employment at such institutions problematic and as such a program’s overwhelming tendency to place their graduates at such places might reasonably for many potential graduates militate against viewing their TT job prospects as equal to those at programs with comparable placement rates into secular institutions. This, I claimed, becomes especially important when considering that some (but certainly not all) overtly religious schools at either the institutional or cultural level explicitly prohibit or actively encourage the condemnation or marginalization of those violating or perceived as violating the religious beliefs or doctrines upon which the school’s mission is in part based. This isn’t some anti-Fordham, anti-Catholic, or even anti-religious screed, and I find it remarkably uncharitable to see it as such.

    Like

  6. Carolyn Dicey Jennings Avatar

    Some interesting points here, although I disagree with the very last point. Perhaps I should point out that Fordham and UCLA come out as similar only for a particular method and time frame, which I use here as an exercise. The method that I used for my full data set, which I think does a better job with respect to time frame, actually puts UCLA ahead by quite a bit. Nonetheless, many departments that are not ranked by the PGR come out as strong in that accounting.

    Like

  7. Jacob Archambault Avatar

    I received my M. A. from the University of Houston, Dr. Mag Uidhir’s home institution, in 2011, the same year Dr. Mag Uidhir was hired (hence, though we do not know each other, we know and have worked with many of the same people). I am currently a Ph. D. candidate at Fordham University.
    There is an underlying tension in Christy’s responses above that, since I think it shared by a large constituency of academic philosophers, it will be useful to bring it to light.
    Christy begins one of his posts above with the concession that only one of the schools a Fordham student was hired at required an explicit statement of faith. He then goes on to state that Fordham “overwhelmingly” places their students at “overtly religious” institutions, and that “75% of recent Fordham grads placed into overtly religious schools”. Earlier in the same post, he defines an overtly religious institution as one “having a mission statement explicitly reflective” of a “certain Christian denomination.”
    If, as the most straightforward reading suggests, the 75% number is meant to reflect Fordham’s overall placement rate, it amounts to the claim that the only places Fordham places their graduates– presumably because it is the only place they can place them – are “overtly religious institutions”.
    If, on the other hand, it is meant to convey that 75% of those placed are placed at religious schools in the sense defined by Dr. Mag Uidhir above, then it is still incorrect and, perhaps more importantly, unjustified. Concerning the first, of 15 TT or comparable placements, 10 (66%) are at schools with some nominal religious affiliation, and 7 of these are at Catholic institutions. This is, of course, quite different from placing students overwhelmingly at overtly religious institutions. I don’t, for instance, know anyone, Christian or otherwise, that would characterize Boston College in such terms.
    More importantly, if Christy is like most of us on this, he simply has no idea whether the schools at which Fordham students have been hired are overtly religious schools, since he likely knows little to nothing about those institutions: he simply assumes it as an explanans for Fordham’s comparatively high numbers.
    And this is just the problem. One simply assumes that the there is some hidden element, determining them to get the posts they do, and demanding that the data capture that. Carolyn’s omission of information concerning distinctions in kinds of TT jobs may be ‘actively misleading’ and ‘almost criminal’, and yet she isn’t obligated to factor these distinctions into her data; Fordham’s placement record success is ‘significantly narrowed in a way that UCLA’s is not’, though the schools at which UCLA places its grads are not ‘better’ jobs; Fordham’s placements are not worse, even though those placements are into schools some of which may ‘actively condemn or marginalize’ things like ‘certain racial affiliations.’ How are we not supposed to get the impression here that a program’s placing graduates into schools with some religious affiliation somehow counts less than placing the same graduate in a secular institution, when the erasure of this distinction is misleading? When doing so suggests that the university has less ‘placement record potential’? When the schools into which these placements are made are, by a series of associations, prima facie made to look like Bob Jones University in the 1970’s?
    It’s rather easy to be ignorant of the fact that to the degree that Fordham’s placement record is dependent on hiring networks of religious schools (and given standard perceptions of Catholicism in e.g. 7th day Adventist and evangelical circles, I’m only really inclined to grant this at most for placements of Fordham students into Catholic institutions, i.e. 7 of its 15 placements, or 47% – and even there I think it’s success has more to do with a shared broadly humanistic conception of education than religion, either a particular one or in general, as such), this is in part (though, I submit, not totally) a function of the kind of soft oppression inherent in the kinds of mischaracterizations above, whereby it is excluded from more secular networks.
    Something quite unfortunate about all of this is that those who serve to be harmed the most from this kind of stereotyping are non-religious students at nominally religious institutions like Fordham. One cannot safely put an ‘atheist buyer beware’ sign around the neck of schools like Fordham and pretend this very action were not perpetuating the very thing one is warning against.
    Maybe it’s about time we change that.

    Like

  8. philosopher Avatar
    philosopher

    I think many do not see Boston College as you do. Here is a statement from the College’s webpage.
    “Jesuit, Catholic Tradition
    finding god in all things
    Boston College is committed to maintaining and strengthening the Jesuit, Catholic mission of the University, and especially its commitment to integrating intellectual, personal, ethical, and religious formation; and to uniting high academic achievement with service to others.
    Jesuits are active in all aspects of University life with a community of more than 150 Jesuits serving as faculty and administrators or studying for advanced degrees within Boston College’s various schools. Members of the Jesuit community also offer Ignatian retreats and spiritual direction to faculty, staff and students.
    Gifts from the Jesuit community have helped establish the Jesuit Institute and the Center for Ignatian Spirituality at Boston College. The Institute sponsors personal research, academic exchange and collective inquiry about the issues that emerge at the intersection of faith and culture. The Center helps members of the University community to understand and implement Jesuit/Ignatian traditions and promotes conversation among other religious traditions represented at Boston College.”
    http://www.bc.edu/about/tradition.html

    Like

Leave a comment