New APPS

Art, Politics, Philosophy, Science. Mostly Philosophy.

recent posts

about

    Why I Will Be Contributing Rankings to the Gourmet Report

    by Eric Schwitzgebel

    I have been asked to be an evaluator for the 2014-2015 edition of the Philosophical Gourmet Report. Contrary to what seems to be the general (but not universal) sentiment of New APPS contributors and commenters, I support the rankings and will participate.

    The PGR rankings have at least three related downsides:

    1. They perpetuate privilege, including the privilege of people with social power in the discipline, the privilege of people in PhD-granting institutions over other types of institutions, and the general privilege of Anglophone philosophy and philosophers.

    2. They reinforce mainstream (“Gourmet ecology“) valuations of topics and approaches, in a discipline where the mainstream needs no help and it would probably be productive to push against the mainstream.

    3. They risk blurring the distinction between second-hand impressions about reputation (especially outside evaluators’ own subareas) and genuine quality.

    In light of these downsides, I understand people’s hesitation to support the enterprise.

    I view the rankings as an exercise in the sociology of philosophy. The rankings are valuable insofar as they reveal sociological facts about how departments, and to some extent individuals (especially in the specialty rankings) are viewed by the social elite in Anglophone philosophy — by the people who publish articles in journals like Nous and Philosophical Review, by the people who write and are written about in Stanford Encyclopedia of Philosophy entries, by the people who teach at renowned British and U.S. universities like Oxford, Harvard, and Berkeley. As a part-time sociologist of philosophy interested in patterns of esteem, I am curious how people in this social group view the field, and I regard the PGR as an important source of data.


    The PGR is thus valuable in part because sociological and historical knowledge about academia in general is valuable. It is sociologically interesting, and of historical interest, to know what sort of esteem Australian universities have in mainstream Anglophone philosophy. It is sociologically interesting, and of historical interest, to see the shifting patterns of social power among Ivy League universities and large public U.S. universities that are able to hire renowned professors.

    The PGR is also practically valuable because knowledge of the centers of social power is practically valuable. To the extent a student wishes to tap into the centers of social power to increase her likelihood of finding a research-oriented job, she should know where those centers of power are; and students and their advisors who are not currently near centers of power might not find it at all obvious where those centers are. By empowering outsiders with knowledge — especially the knowledge that renowned universities like Harvard, Oxford, and Yale might not be the best universities in their subfield — the PGR to some extent works against the perpetuation of privilege, despite the fact that it reinforces privilege in other ways. Also, to the extent one wishes to fight against mainstream perceptions of the discipline, it is of interest to track what those perceptions are and how they are changing over time — though if this were one’s primary motivation, one would probably oppose the PGR. Finally, to the extent one respects the judgment of philosophers in the Anglophone philosophy mainstream, one might infer differences in real quality from differences in reputation.

    On the last point: If you think that Anglophone philosophy mainstream judgment is grossly erroneous in general, you might reasonably infer that the PGR does more harm than good; but I don’t hold that view myself. In philosophy of mind, for example — my own specialty — I think that the best-regarded philosophers tend in fact to be excellent philosophers who deserve their good reputations.

    One area in which I think mainstream philosophical judgment is ill-tuned is in its disregard of non-Western traditions. However, I believe that the PGR has the potential to be progressive on this issue. For example, in treating Chinese philosophy as an area worth special remark, despite the small number of PhD-granting philosophy departments in Anglophone countries who have specialists in the area, it gives the subarea more visibility than it otherwise would have. And were there sufficient hiring in other non-Western traditions, I suspect the PGR would adapt to reflect that.

    Despite my support of the PGR rankings, I think it is important that the rankings be viewed critically, as a rough tool for revealing certain sociological patterns in the discipline. I would very much like to see other approaches to evaluation, which would help put the PGR rankings in context as only one way to think about the social structures that drive academic philosophy.

    Posted in ,

    37 responses to “Why I Will Be Contributing Rankings to the Gourmet Report”

    1. Christopher Gauker Avatar
      Christopher Gauker

      Eric, you do not know enough about the work of the people you are evaluating to be able to do it competently. When you assign a low number to a department, and you don’t have a lot of first-hand knowledge of the work of those people, you are deliberately harming those people for no good reason. You should not do it.

      Like

      Reply
    2. Luke Maring Avatar
      Luke Maring

      Eric:
      The whole ’empowering outsiders with knowledge’ argument is kind of fraught. A number of people have pointed out that if conveying knowledge is the important thing, there are better ways to do it. If you’re trying to empower grad school applicants, tell them what places have good placement records into TT jobs. Tell them what kinds of schools those TT jobs tend to be in. Tell them how long it usually takes to graduate and what kind of financial support they can expect.
      If you want to empower newly-minted PhD’s on the job market, well, forget the whole business. Given the state of the job market, just about everyone I know has applied to every job they are qualified for. The vast majority of us, even if empowered as you envision, aren’t in a position to be choosy. The ones who are already know which departments have good reputations.
      So who, exactly, are you hoping to empower here?
      That leaves the argument that the PGR is an interesting piece of sociological data. That might be right. But when collecting the data has the downsides you yourself mention (perpetuating privilege, reinforcing mainstream values, and blurring the line b/w quality and reputation), I struggle to see personal curiosity as a sufficient reason go ahead.

      Like

      Reply
    3. Catherine Kemp Avatar
      Catherine Kemp

      I can support this project IFF (1) the PGR is retitled Philososopher’s Social Register and (2) its sociologists do full ethnomethodologies every time they observe as well as participate in its rituals.
      This is the most honest defense of the PGR I’ve seen so far, the objections already noted here aside. I’m curious about whether other people putting in Jenkins’ 50+ think all that effort is worth it, seen in this light.

      Like

      Reply
    4. Catherine Kemp Avatar
      Catherine Kemp

      That is to say, Ichikawa’s 50+!

      Like

      Reply
    5. Anonymous Grad Student Avatar
      Anonymous Grad Student

      Perpetuating privilege is not just a “downside”; it’s something that we should seriously consider as a moral reason that might override things like sociological interest.
      I also just don’t buy the argument that the PGR, in some ways, works against privilege. Privilege influences the knowledge that the PGR produces regardless of whether, by reading the PGR, I draw the conclusion that I should attend Ohio State over Harvard for some specific research interest. If anything, this seems more like generating an elite for each subfield in addition to the overall elite, which just makes elitism worse.

      Like

      Reply
    6. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Christopher and Catherine: I recognize that I am not competent to directly evaluate scholars of, for example, early modern philosophy. To the extent my ratings will reflect the presence of those people, it will be based on my sense of their general reputation. Historians’ reputation in the “Gourmet ecology” community as a whole might not correspond well with the real quality of their work. I leave that as an open question, which is why I prefer to conceptualize the survey as primarily an exercise in sociology of philosophy.

      Like

      Reply
    7. Anonymous Grad Student Avatar
      Anonymous Grad Student

      Eric: Thanks for your post, though I strongly disagree. I think the real worry about the PGR, despite what you’ve said above, is that people are just not able to take it with the grain (or spoonful) of salt that it really should be taken with. This is evidenced by the fact that, until recently (and perhaps still), the PGR has held so much power and influence in professional Anglophone philosophy. I do not think it is much of an exaggeration to say that some (or many?) in the profession regard or treat it as “gospel,” or at least as much, much more than merely a source of interesting sociological information (as you suggest). The fact of the matter is that the methodology (statistical and otherwise) of the PGR is bunk, and that alone deprives it of the potential of being a source of much interesting, or at least useful, information at all.
      If people really regarded, or were going to start to regard, the PGR as a sociological experiment and nothing more, then I don’t think there would be much of an issue. But at least up until now, it has not been regarded as such. It is all too easy, unfortunately, to slip into thinking of the PGR as a legitimate rankings system. And I highly doubt that, if it continues, philosophers in general will be any more successful at viewing it as merely a sociological experiment.

      Like

      Reply
    8. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Luke / Anon Grad 13:49: When I applied to graduate school in 1990-1991, the PGR was new and known only to a few and did not inform my own application process. Despite the fact that I was coming from a position of privilege as an undergraduate at Stanford, I had little idea what departments were perceived as generally weak or generally strong and which departments were regarded as strong in the areas I was most interested in. I could not read, much less evaluate, a sample of representative work of philosophers from a wide range of departments; so my application procedure was pretty haphazard, based mostly on my general overall sense of the quality of the university and based on a few stray sentences from advisors I was too shy to ask for detailed advice from.
      It might be suggested that students and their advisors (to the extent students are assertive enough to ask and the advisors forthcoming) should ignore mainstream perceptions of reputation and reach their own independent quality judgments. Or it might be suggested that linear scales of reputation are problematic. I have some sympathy with both concerns. But I do also think that it empowers students to know the sociological facts about reputation that are reflected in the PGR. It’s hard for me to see ignorance of these facts, which have such an impact on job prospects among other things, as other than disempowering. So I really do think that there are empowerment / privilege perpetuation issues on both sides of the argument.

      Like

      Reply
    9. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Anon Grad 14:40: I agree that people often fail to view the PGR results as skeptically and critically as they should, and that this is a substantial problem. I see two solutions. One is to emphasize, as I do in this post, that it is a sociological measure that is only imperfectly and disputably related to genuine philosophical quality (which itself is a tricky concept). The other is to encourage a variety of measures, each of which might partly support and partly undercut the others, providing a context in which it is easier to recognize the limitations of any one measure.

      Like

      Reply
    10. David Hunter Avatar
      David Hunter

      Eric, I don’t understand your reply to Christopher and Catherine. You say that when you are not competent to assess a philosopher’s quality, you will report based on your ‘sense of their general reputation’. Set aside whether you or anyone else is a reliable judge of anyone’s general reputation. But even if you were, this would make the PGR, not a reputational survey of quality, but a reputational survey of reputation. Is data on reputed reputation of even sociological interest?

      Like

      Reply
    11. Luke Maring Avatar
      Luke Maring

      Eric:
      I don’t think that response is sufficient. If we faced a choice between the PGR and no guidance at all, it might be. The PGR would be necessary for an important good, and the value of that good would have to be weighed against its disvalue. But we are obviously not faced with that false dichotomy. Again, as many have pointed out, data not provided in the PGR would serve potential graduate student’s interests just as good or better: data on placement rates into TT jobs, on where those TT jobs tend to be, on what sorts of things different departments have a strong publishing record on.
      So again: it seems that your claim that the PGR is sociological interesting has to bear the weight of argument. And I still, given its downsides, struggle to see sociological curiosity as a sufficient reason to go ahead.

      Like

      Reply
    12. Cynic Avatar
      Cynic

      If the PGR’s value is measured by its sociological interest, then it should be conducted by sociologists who have minimal interest in seeing certain results, not by philosophical insiders who simply reproduce the very privilege they claim to be putting in check.

      Like

      Reply
    13. Anonymous Grad Student Avatar
      Anonymous Grad Student

      I’m Anonymous Grad Student at 13:49, and thanks for your response.
      I sympathize with the concern that some students are too shy to seek out professors’ advice about these matters, but I would hope that professors who agree to be letter writers for students also offer to help them decide which schools to apply to, and how to evaluate options. I just think that real human beings are better positioned to help evaluate, on a case-by-case basis, what schools would be good fits for certain students.
      I’m also not convinced that knowing what a fixed group of carefully selected philosophers perceive to be the case about such and such a school is valuable knowledge for prospective graduate students to have. Not all sociological facts are valuable, and many philosophers have given good reasons to think that PGR facts are not valuable and in some sense harmful.
      The best advice I ever received was this: “You should only do a PhD in Philosophy if you will enjoy the process of earning the PhD. It should be valuable to you even if you don’t land a TT job.” Unfortunately, PGR facts tell me nothing about how I can best predict whether I will enjoy my time in graduate school to make the experience worth the risk of not landing a TT job. Things that make my institution (though not PGR-approved) fit the bill: geography, the size of the department, faculty support (including a genuine interest in my work), the climate for women, the climate for philosophers interested in feminist philosophy, excellent external funding (that I could not take with me to an American school), and so on. These are things that prospective students can learn by contacting current graduate students in departments.

      Like

      Reply
    14. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      David Hunter: I think the reputation of someone in the PGR ecology is partly constituted by what people in the PGR ecology think the reputation of the person is. I hope that doesn’t seem too paradoxically self-referential!

      Like

      Reply
    15. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Luke Maring: Data on placement rates is a good factor to consider, and it’s good to see Carolyn Jennings working on this; but it is complicated and methodologically difficult due to inconsistency and non-transparency in departmental reporting of non-placed students and dropouts, as well as the varying factors that might influence those. Undergraduate applicants evaluating “what sorts of things different departments have a strong publishing record on” seems a lot to hope for unless they start with something like the PGR or NRC. I think it’s good if they can combine various sources of knowledge to arrive at a final judgment — e.g., by checking out departments rated high in their target specialty, as well as departments recommended by their advisors, and then examining the websites and some articles by the relevant professors at those universities. They might also consider a variety of other factors such as location. They might find their opinions at the end of the process to justify applying to a very different set of departments than they might have done if they had considered only the PGR rankings; or they might not.

      Like

      Reply
    16. Mitchell Aboulafia Avatar
      Mitchell Aboulafia

      When I first read Schwitzgebel’s post I thought it might be a satire that failed to find its natural title, “The Empire Strikes Back.” But then I looked more closely and concluded that Eric appears to be serious. And if so, I want to thank him. He’s done a wonderful job at demonstrating why it would be wrong to fill out the evaluations. It is clear from his response that he has not done his homework regarding the host of methodological problems that the PGR faces, including the PGR’s sham claim to anonymity regarding surveyed departments (due in part to Leiter’s obsession with telling us about where high profile people are currently located). Let’s check out a couple of Eric’s statements.
      “I view the rankings as an exercise in the sociology of philosophy. The rankings are valuable insofar as they reveal sociological facts about how departments, and to some extent individuals (especially in the specialty rankings) are viewed by the social elite in Anglophone philosophy…. As a part-time sociologist of philosophy interested in patterns of esteem, I am curious how people in this social group view the field, and I regard the PGR as an important source of data.”
      So we now have a public admission regarding what the PGR is really about. “The rankings are valuable insofar as they reveal sociological facts about how departments, and to some extent individuals (especially in the specialty rankings) are viewed by the social elite in Anglophone philosophy.” Okay, but if so, in order to avoid being charged with false advertising, this fact about the PGR should be stated in bold at the top of every rankings page that the PGR posts, instead of pretending that the PGR provides an informative overview of the profession. And please note how Eric appeals to what he calls the sociology of knowledge here, but only a strangely compressed notion of the field. Folks interested in the sociology of knowledge, along with other sociologists, often examine how networks of influence and power are formed and perpetuated, not to justify or merely to highlight them but often to show how they are illegitimate or how they lead to confusions regarding notions of merit, worth, value, etc. The problem we face in Philosophy and other fields is how merit often gets entangled and confused with power, prestige and yes, money. If we are going to appeal to sociology, let’s go all the way! I’m sure that the sociologists could provide some interesting commentary on the PGR. (No doubt the sociologists of knowledge would just love Eric’s view of their enterprise, and will at once turn their efforts into producing a cornucopia of information regarding centers of power, for the benefit of the next generation, which can be empowered by locating these centers more easily in order to seek them out.)
      There are other issues, but here I’ll focus on just one.
      “On the last point: If you think that Anglophone philosophy mainstream judgment is grossly erroneous in general, you might reasonably infer that the PGR does more harm than good; but I don’t hold that view myself.”
      A classic straw man, in this case, one seeking to drum up support from the so-called Anglophone mainstream, which may feel attacked by the criticisms of the PGR. First, we don’t know if we are getting an accurate picture of the so-called Anglophone mainstream due to methodological problems with the survey. Second, this is not about criticizing the judgments of philosophers practicing philosophy in a certain fashion. It is about a Logic 101 fallacy, Hasty Generalization. It is about taking the judgments of a group of philosophers–whose merits within their own domains is not being challenged–and generalizing them to the profession as a whole, whether in the form of overall or specialty rankings. Third, the rhetorical use of the phrase “grossly erroneous in general” reflects a strategy I have now seen employed several times by defenders of the PGR, namely, making preposterous claims about the views of opponents of the PGR–they are people who completely undervalue or see no merit in what we do–thereby providing grounds, psychological if not philosophic–for dismissing the opponents and their claims.
      I do not infer that the PGR does more harm than good because I think “Anglophone philosophy mainstream judgment is grossly erroneous in general.” I think the PGR does more harm than good precisely because it doesn’t recognize or acknowledge its own limitations, and it is cavalier about its methodological faults.

      Like

      Reply
    17. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Cynic: It would be nice to see collaboration between sociologists and philosophers on methodology.

      Like

      Reply
    18. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Anon grad 13:49 / 15:41: “You should only do a PhD in Philosophy if you will enjoy the process of earning the PhD. It should be valuable to you even if you don’t land a TT job.” I agree with that, and I give similar advice in my series of posts at The Splintered Mind on applying to grad school. Most students would do well, I think, not only to consider reputation, as imperfectly measured by the PGR rankings, in deciding where to attend graduate school, but also to consider the other types of factors that you mention. Of course, the existence of the PGR doesn’t prevent students from considering those things.

      Like

      Reply
    19. Annoyingly Pedantic Sociologist Avatar
      Annoyingly Pedantic Sociologist

      I have to disagree that the PGR is itself an example of the sociology of philosophy. If this were the case would it not be better done by those who are sociologists of philosophy? Randal Collins or Martin Kusch?
      Like other bureaucratic and administrative exercises in evaluation or ‘audit’ – the UKs REF, the Care Quality Commissions – the methods of the social sciences, or things very much like them, are being used. But this does not make them exercises in sociology. The use of social scientific methods is the foundation of civil service bureaucracies and, insofar as this is the case, the foundation of the modern nation state. This does mean the UK’s civil service is engaged in the sociology of the UK.
      Certainly we can apply a sociological imagination to all of these phenomena and doing so allows us greater insight into the social consequences of these endeavours. Certainly the PGR is engaged in the construction and reconstruction of social facts about philosophy – or a particular institutionalisation of philosophy. However, the PGR itself is an aspect or ‘social fact’ about this particular institutionalisation. As such rather than ‘uncovering’ hitherto fore unknown facts it would be more accurate to say that it is a reflection of those facts. The recent debate over Leiter’s positions seems as good an example as any.
      None of this is to suggest that the PGR does more harm than good – or vice versa – or that it should be abandoned or not. It is just to be clear that exercises like the PGR are no different to other forms of audit merely because they happen to be done by academics.

      Like

      Reply
    20. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Mitchell: I would be interested to see a thoughtful critique of the state of mainstream Anglophone philosophy, including measures of reputation, from sociologists of knowledge. I might even agree with the conclusions of such a critique. An important part of my own philosophical research is concerned with calling into doubt how much knowledge and expertise professional philosophers have in their target areas of specialization (esp. about consciousness and about evaluating moral dilemmas).

      Like

      Reply
    21. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      APS: “Rather than ‘uncovering’ hitherto fore unknown facts it would be more accurate to say that it is a reflection of those facts.” I don’t disagree. The social facts are (more or less) already known to the evaluators; the PGR displays those social facts for a broader audience.
      “It is just to be clear that exercises like the PGR are no different to other forms of audit merely because they happen to be done by academics.” I don’t think the fact that the PGR is conducted by academic philosophers gives it special status. REF, NRC, etc., also have value — and their own sets of problems.

      Like

      Reply
    22. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      I need to take a break from approving and responding to comments, to focus on other things. So please be patient: If you submit a comment, it might not show until tomorrow.

      Like

      Reply
    23. Catherine Kemp Avatar
      Catherine Kemp

      Thank you for being so generous with your responses, Eric, and for taking the time to moderate this thread. The discussion has been both revealing and quite fruitful.
      I think what’s emerged here is that you other supporters see in the PGR what Dan Doctoroff in 2013 called a “virtuous cycle” at NYU. In his response to critics of NYU’s housing subsidies for faculty that run at times to second homes upstate, he describes the desirable effects of aggressive recruitment of top talent. This is one of these effects:
      “[The University’s) Ph.D. programs are steadily rising in nearly every academic discipline.”
      http://observer.com/2013/07/from-n-y-who-to-nyu-president-sexton-compensates-top-talent-lavishly-but-not-foolishly/
      I am certain that a very important measure of the philosophy faculty at NYU is its standing in Leiter’s rankings. This raises the alternative justification that many have offered for the PGR: its importance for the audience of deans and provosts who authorize resources, including new tenure-stream lines and the clusters of perks that attend them, for faculties that can parade their rank when making the case for new lines in philosophy, rather than history, english, sociology, or political science.
      One can also see here why some faculties that have done well in the PGR years may find it hard to give it up, and why so many people are holding their noses and continuing to support this brand of rankings: it’s not just people who have arrived, as it were, but those who can imagine themselves making it into the ranks of the highly-ranked one day, who are hoping the Brogaard co-editorship calms the nervous participants and muffles the noise of the skeptics.
      The risk, of course–even if the PGR brand survives the damage Leiter has done to it already–is that some day some english, or more likely sociology, or political science, or math faculty will discover how flimsy the rankings methodology in philosophy is, and make that flimsiness itself part of its own case to the dean, or the provost, for more lines, or more perks, as against the claims of the resident highly ranked PGR department. How embarrassing!

      Like

      Reply
    24. Derek Bowman Avatar
      Derek Bowman

      “You should only do a PhD in Philosophy if you will enjoy the process of earning the PhD. It should be valuable to you even if you don’t land a TT job.”
      This is true, but it is far too weak a criteria. Even if this is true, you probably still shouldn’t pursue a PhD in philosophy. Enjoying yourself for 5+ years is great, but unless you’re independently wealthy, you do have to find a way to make a living when it’s over and done with.
      But at best rankings information (even of the sort Carolyn Jennings is working on) can only affect the distribution of positional advantage in a broken graduate education and employment system. It does nothing to help (prospective) graduate students and (future) job applicants overall.

      Like

      Reply
    25. Concerned outsider Avatar
      Concerned outsider

      ES writes, “The rankings are valuable insofar as they reveal sociological facts about how departments, and to some extent individuals (especially in the specialty rankings) are viewed by the social elite in Anglophone philosophy — by the people who publish articles in journals like Nous and Philosophical Review, by the people who write and are written about in Stanford Encyclopedia of Philosophy entries, by the people who teach at renowned British and U.S. universities like Oxford, Harvard, and Berkeley.”
      One concern that I and many others have about the PGR is about how the evaluators are selected. If they were selected by, say, simply inviting every author who has published in Nous and Phil Review in the last ten years to evaluate for the PGR, that would be better than the current system, to my mind. At least, it would be a more strongly anchored selection principle than the current one, which seems to be to conjure up names ad hoc, resulting in an unrepresentative, privileged subset of people who imperfectly overlap with the social elite as defined by publishing in top journals, SEP representation, or employment at a renowned university. Lots of people who have published in the elite journals, including these two journals, are not asked to participate. Certain senior, esteemed, highly accomplished, and up-to-date specialists in even LEMMING fields at highly PGR-rated departments have, according to their own first-person reports that I have heard, never been invited to evaluate. Similarly, many people who write for the SEP and are written about in the SEP have not been invited to evaluate–at least that is my (perhaps mistaken) understanding. At the same time, other people who would seem at first blush to not be as much a part of the social elite (such as junior people who have made an impression on the powers-that-be at the PGR) are invited.
      If this is true, then even if we focus on the social elite, and even if we dismiss the exclusion or marginalization of certain areas, institutions, and individuals, we aren’t getting a representative sample of the social elite–as measured by the standards of top journal publications and departments–by the current selection procedures. A better method–if the aim is to measure the views of social elites, where ‘elite’ is operationalized in the way suggested by the quote above–would be to offer evaluator invitations to anyone who has published in certain journals, has been written about in the SEP or written for the SEP, or is one of a certain set of departments.

      Like

      Reply
    26. Bharath Vallabha Avatar
      Bharath Vallabha

      Eric, should there be fashion magazines with pictures of just ultra skinny, young white women? Certainly one can defend this practice by saying that ultimately it provides sociological information about ideals of beauty in the 21st century. But then with the sociological defense pretty much anything can be defended as providing sociological data. Should misogynistic gamers spew hatred towards Sarkeesian? Why not, as long as it provides sociological data about the extent of the misogyny? Sometimes claiming to merely descriptively capture a state of affairs, without aiming to endorse it, can itself be a way of endorsing it.

      Like

      Reply
    27. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Catherine: Thanks for that thoughtful reply. I agree that the PGR does sometimes play the role you suggest. I agree that it has substantial methodological shortcomings. I also think that the NRC, REF, and other attempts at qualitative / reputational measures also have substantial shortcomings. It’s easy to think of quantitative measures like citation counts as more valid than the subjective opinion of recruited middle- to high-status people in the mainstream of the field. In some respects, those methodologies are better, but I don’t think they’re decisively better. The litany of legitimate complaints against citation counting, for example, is long.
      For this reason, I would encourage the proliferation of a wide diversity of measures, with complementary methodological shortcomings to the extent that is possible — measures played off each other so that people don’t take any one measure too seriously.

      Like

      Reply
    28. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Concerned outsider: You write: “One concern that I and many others have about the PGR is about how the evaluators are selected. If they were selected by, say, simply inviting every author who has published in Nous and Phil Review in the last ten years to evaluate for the PGR, that would be better than the current system, to my mind.”
      I agree. The invitation system is non-transparent and possibly biased. I am inclined to think that your suggestion would be preferable to the current system. Or maybe one improvement on it would be to try to come up with a short list of journals that are highly visible by objectivish criteria (Nous and Phil Review might win, but there are other contenders). Another possibility might be random sampling from disciplinary societies in the target countries — though that risks amplifying the Anglophone bias.

      Like

      Reply
    29. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      Bharath: You write: “Sometimes claiming to merely descriptively capture a state of affairs, without aiming to endorse it, can itself be a way of endorsing it.” I agree. So I think it’s reasonable for people who think that the rankings are either grossly misleading or socially egregious should oppose the rankings. I think they’re likely to be somewhat misleading and that they have a mix of social upsides and downsides, and only because I have a moderate view on those questions do I think it makes sense to put forward an argument in favor on sociological grounds.

      Like

      Reply
    30. Catherine Kemp Avatar
      Catherine Kemp

      Eric, okay, but if we all, in higher ed, were really in the same boat on the issue of judgments about quality and the rationale for allocation of resources, it would be the students and their parents, instead of other departments, coming with pitchforks. Your response suggests that we all throw up an assortment of measures and not take any one too seriously–think how that would sound to someone paying off what is, in effect, the mortgage on my summer house in the Adirondacks.

      Like

      Reply
    31. ck Avatar
      ck

      Thanks for taking the time to reply to everyone in detail, Eric. Here’s another.
      You write that part of your response to some of the shortcomings you see in the PGR is to “encourage a variety of measures, each of which might partly support and partly undercut the others”.
      But are you actively doing anything to encourage other measures? And doesn’t it concern you that Leiter actively and sometimes even abusively tries to discourage any alternative measures to his (such as Alcoff’s project)? Don’t you think that by allying yourself with Leiter’s measure you are not only failing to actively encourage a variety of measures but also indirectly discouraging such measures? Do you really want to encourage such measures, or do you want to see yourself as someone who would encourage them?
      You also write: “I would be interested to see a thoughtful critique of the state of mainstream Anglophone philosophy, including measures of reputation, from sociologists of knowledge.”
      But how serious are you about this? Are you active about your interest? Have you read Randall Collins? What about Neil Gross’s sociological account of the profession in his socio-biography of Richard Rorty? If you were interested to see such thoughtful critiques, you probably would have seen (and read them) by now. So I find your statement confusing. Are you really interested? Or do you just want to see yourself as interested?
      This leads, I suppose, to another form of the same question: Are you really in a good position to evaluate the quality of philosophy programs on the whole or do you just want to see yourself as someone who is? And is Leiter really in a position to pick good evaluators of philosophy programs or does he just like to see himself as someone who is thus positioned?

      Like

      Reply
    32. Eric Schwitzgebel Avatar
      Eric Schwitzgebel

      CK: I have read Randall Collins’ Sociology of Philosophies cover to cover. I have read multiple works by Martin Kusch and Neil Gross, including Kusch’s Psychologism and Gross’s book on Rorty. I arranged for Gross to speak to the philosophy department at UCR and we have had email discussions about the politics of philosophers in connection with my data on the voting patterns (and party registration) of professional philosophers (Schwitzgebel and Rust 2010). I have posted many quantitative analyses of social patterns in philosophy on my blog, The Splintered Mind, adapting analytic tools from sociology and bibliometrics. I have developed two prestige ranking systems for philosophers, one based on citations in the SEP and one based on name occurrences in PhilIndex abstracts. Brian Leiter posted an analysis of departments based on my SEP rankings of individual philosophers, which I noted on my blog better correlated with the PGR department rankings than the NRC rankings. I have published extensively on how social and/or non-rational factors can have a major influence on philosophers’ professional judgments in their areas of expertise, especially regarding consciousness and moral dilemmas. On The Splintered Mind, I have defended what I call the “Strong Program in Psychology of Philosophy” drawn with intentional parallel to the Strong Program in sociology of knowledge.
      So: Yes, I am serious. I understand why you might have been inclined to think my expressed interest in sociology of philosophy was merely a pose, if you don’t know my work.
      (Sorry if this comes across as too defensive. My morning coffee might not yet quite have kicked in!)

      Like

      Reply
    33. Thomas V. Cunningham Avatar
      Thomas V. Cunningham

      Luke/Anon 13:49/ES: When I applied to grad school in 2003 & 2004 the Phil Gourmet empowered me in a way that may serve as a small data point relevant to ES’ claims re: privilege. I was a late-comer to philosophy. I spent the first half of my undergrad studies in the sciences. I completed my Phil BA quickly, with a highly impacted course load, while also working toward another degree. So I did not get to know many profs well and was not well positioned to have lengthy discussions about where to apply to, how to best present myself, etc. Moreover, what advice I did receive was widely divergent: some said do not apply at all — go be a lawyer! — while others said try MIT, Columbia, or Duke. In the end I was selective in my applications (per what I think was bad advice) and I was rejected across the board. When applying the following year I was able to use the PGR to guide me in looking across the US and beyond to find places worthy of my consideration, given my interests and aptitude. In my case, the PGR did not prejudice me against lower-ranked depts. Rather, it gave me context that is available only to the already-initiated, which I was not. From this context, I was able to make informed choices about applying while trying to balance my interest in professional philosophy against other life goals, like making livable wage, having any control over where I lived in the world, etc.
      So, this is just to say, YMMV, but in my case I was an outsider of sorts* trying to figure out how best to understand the philosophy ecosystem and PGR provided important information to me that was otherwise unavailable or so widely divergent as to seem random from the other information sources I had access too. And I was able to process the information knowing it was biased. I did not read the PGR and then, thereby, exclude schools where I might fit in. But I did learn from it what “the establishment” thought of those schools.
      *I am a white male who went to a fine public university (UCSD), so I am clearly an insider, or privileged, or whatever, in many ways relevant to the sociology of knowledge, philosophy, etc. But at the time I applied to grad school I was not an insider and did not have insider knowledge vis understanding anything about the relative merits of philosophy departments I hoped to gain entry to, which is the data point I aim to convey at the moment.

      Like

      Reply
    34. Luke Maring Avatar
      Luke Maring

      Thomas V. Cunningham:
      Those are all fair points. But my claim is not that the PGR is useless to potential grad students–hell, I used it when I applied. My claim is that we can’t validly defend the value of the PGR by imagining a world where there is no guidance whatsoever. The question is: what else would be helpful to grad students, what (if any) value does the PGR provide that those other things won’t, do those other values (again, if there are any) sufficiently outweigh the obvious disvalue of the PGR.

      Like

      Reply
    35. Thomas V. Cunningham Avatar
      Thomas V. Cunningham

      Luke: You have a fair point. As I see the issue, commentators differ w/r/t what the aim of the PGR is and what relative (dis)value it has, given its aim. There is disagreement over whether it aims to (a) inform potential entrants to the field of Philosophy (b) serve to promulgate a real or imagined hegemony on the part of anglophone analytic philosophy vis other parts of the field, (c) identify the privileged in philosophy and serve as a means for policing entrance into this social group/profession (related to [b]), (d) provide a means for Brian Leiter to advertise himself and increase the value of his brand, or (e) any number of aims.
      My view is that discerning the merits of the PGR requires an awareness of the disparate aims and usages it has. Personally, I think that its primary aims have been (a) and (d). However, depending upon how one interprets BL’s various offensive and inoffensive comments on his blog and elsewhere, it is used for (b), (c), and (e). Now, the question is to what extent the harms or benefits (depending on your POV) of the way the PGR is used provide reasons for or against its continued production, relative to the aims (a) and (d).
      Personally, I think that BL has succeeded in advertising himself through PGR, his blog, his academic work, etc. That ship has sailed. And it is not returning to port. The PGR may easily persist separate from this aim (e.g. through new leadership, collective or otherwise). So all that remains is balancing the putatively negative effects of the PGR with the aim of informing potential applicants. This should hinge on intuitions about what applicants are like and the type of ecosystem they currently operate in when striving to become a professional philosopher (or academic with a Phil PhD). My knowledge of this is mostly based on personal experience, anecdote from colleagues, and scant familiarity with work like that done by OP on the chances students have of realizing this goal based on their alma mater. My impression is that philosophy generally rewards the privileged in who is allowed entrance to the field because of status-quo bias: those who talk the talk right are accepted, provided they do not fail the hurdles of GRE, TOEFL, etc. scores and get sufficient recommendations. However, my impression is that many schools spend significant effort attending to the extent to which students appear to be a good fit. So some students may significantly improve their chances of entrance by better understanding this “fit” issue and by tailoring their materials in light of this. However, a Catch-22 lurks here: without access to information, how are they to know this and act accordingly? This is where the PGR comes in. For those students who do not have access to information (because they are awkward, because their profs are awkward, because they work and go to school, because their profs think poorly of them, because they have psychosocial problems, because whatever…), the PGR provides a repository of opinion and a detailed account of its method, such that they can evaluate that opinion in light of the procedures that produced it. Those that are already so privileged that they can understand the debate over the relative merits of the PGR probably do not need it anyway and are probably part of the group that (occasionally?) merely uses it to spur their own ego or inflated sense of self.
      This is the sense in which I continue to agree with OP, that the PGR may serve as counter-point to the privilege that is rampant in a profession that is bizarrely ultra-pretentious and also tolerant of the most outlandish, unpopular, ridiculous views. For this reason, I think the PGR is worthy of preservation as well as reform. I appreciate the argument given well by some and poorly by many that the PGR has become a cudgel wielded by Leiter. The evidence of this has merit. I also appreciate the argument that the PGR serves as a poor method for assessing the competency and fit of job applicants post-PhD. On this point, I am less convinced, since scant good evidence has been given. (I await Dicey-Jennings’ and others’ strong work to evolve so that we get a better vision of what biases there are and how they are propagated. But I am not sure what, if any, affect this knowledge will have on practices, save for individual committee members’ reflectively changing their behaviors in accordance with it.) Finally, I appreciate the argument that PGR is used as a means of distinguishing bad philosophy (continental) from good (analytic). I am unmoved by this argument because it seems to be a sea of straw persons. My limited experience in the field is that there are lots of different ways to make it as a philosopher over a life time. Many people practice many different approaches, and though on the large scale the analytic/continental divide exists, it is not so much a war to be mindful of as a product of various selection pressures as individuals try to make their arguments and continue to be supported as thinkers. Such as it is I don’t really care about it. Typically when I talk to colleagues I learn that most are quite eclectic, ecumenical, and curious in their intellectual pursuits. Many read widely and appreciate different POVs. This may not be the route to fame and fortune as a philosopher (such as there is a route), but I think its very common.
      So, given my assessment of the arguments I don’t see how disbanding the PGR is worth it. At this point, BL has built a platform for influence with or without it. His sounding board is his blog, not the PGR. That will persist regardless of the PGR. And if you think it is an issue, it will remain an issue with or without the PGR. The PGR serves a valuable aim of increasing access to privileged information in a way that is rather transparent about how it is produced. And anyone wanting more information can find discussions like this and others regarding the qualifications of its merit.
      (Now all this aside, I think this is the only reason for keeping the PGR. As a sociological inquiry, I think that while it delivers meaningful data, I agree with the spirit of Anon 16:19 that there are probably alternative methods of data collection that merit priority over the PGR. That is, IF the aim was to research the sociology of philosophy, then I think the PGR is a cumbersome and biased way to do it. Better to do it another way for that aim.)

      Like

      Reply
    36. Eddy Nahmias Avatar
      Eddy Nahmias

      Three brief points in support of Eric (and the PGR):
      1. Like Eric (and unlike Thomas C.) I applied to grad school pre-PGR, and coming out of a non-analytic program as a philosophy minor wanting to do analytic philosophy of mind, I was largely clueless and most of my profs could offer little reliable advice. I also knew little about how competitive the process was (hence got into only two programs of the 10-12 I applied to), so I shot over my head for some of those, and I’d guess that I missed at least a half dozen programs at less prestigious universities that would have been more plausible and useful targets. I got incredibly lucky that I was accepted to Duke (probably because Owen Flanagan looked past my weaknesses for some reason). I’m pretty sure I thought Duke was considered more highly within the field than it was, based on the prestige of the university in general, and I had a wildly over-inflated perception of the chances my peers and I had of ending up with a good tenure-track teaching position (again, I got very lucky after finishing).
      I say all this just to remind people that, we need something in place to provide students a sense of where they should be applying, given their interests, and where they will be more or less likely to get jobs.
      2. So, is PGR what we need? I am not yet convinced things would be better without it . I’m entirely convinced things would be better with more than just it. (I also think the PGR will be better off with different leadership and its new leadership should welcome ‘competitors’ and let the market sort out what’s used). Information about (and some sort of ranking of) placement records is crucial. But doesn’t Leiter and the PGR deserve credit for encouraging (in some cases by publicly shaming) departments to get their placement info up to date and online?
      3. The area rankings in the 3-4 areas I would feel confident ranking are, by my own estimation, highly accurate. I use them to advise my MA students on where to apply, and inductively I then trust the area rankings to provide guidance to students interested in areas I do not know well enough to provide useful info.
      Anyway, thanks Eric and everyone commenting, on providing a useful discussion.

      Like

      Reply
    37. Guilty of Using the PGR Avatar
      Guilty of Using the PGR

      “I would hope that professors who agree to be letter writers for students also offer to help them decide which schools to apply to, and how to evaluate options.”
      Then their professional futures are in the hands of their professors, who may or may not take such initiative. That seems to reinforce power differentials in precisely the way that (as Dr. Schwitzgebel pointed out) the PGR weakens them. I, like him, was in no position at all coming out of my BA to make an informed decision on a PhD program, and the advice I received from my mentors was (necessarily) highly idiosyncratic and vulnerable to individual bias.

      Like

      Reply

    Leave a reply to Eric Schwitzgebel Cancel reply