Choosing a grad school

One of my more popular posts has been Where you get your BS in CS matters, which looked at a report on which universities sent students on to PhD programs in computer science.  Choosing a college for undergraduate studies based on reputation or rankings is always problematic, because the ratings generally measure the wrong thing (usually size, wealth, and reputation, rather than educational quality).

Choosing a grad school is also difficult.  Reputation for research quality matters more for choosing a grad school than for an undergraduate, because grad students are expected to do research and their future careers may be highly affected by the reputation of their advisers or grad schools.  But overall reputation is not enough—reputation within a subfield is what matters most.

An important question for students applying to grad schools to ask is whether the places they are applying to (or are accepted at) will provide a rich environment for the sort of research they want to do.

When I was applying to grad schools (44 years ago), I made the mistake of looking only at overall reputation, and chose the math department at Stanford. The first year in math grad programs consisted of three subjects: real analysis, complex analysis, and abstract algebra (I believe that math grad program have not changed this focus in the past 4 decades).  After I passed my comprehensive exams (3 grueling 6-hour exams), I started looking for a thesis topic and an adviser and realized that no one in Stanford’s math department at the time did the sort of math I enjoyed (discrete math: combinatorics and graph theory).  Luckily, Stanford’s computer science department had four faculty in those fields, and the CS department there accepted my application for a transfer.  As it turned out, I didn’t end up doing combinatorics, graph theory, or any of the mathematical computer science subfields—I flirted with a lot of subjects (including computer music) and my thesis was in computer-aided design for VLSI.

Grad school applicants nowadays don’t have to make my mistake—they have access to much more information and much finer-grained information about grad programs than was easily available 40 years ago.  One resource worth looking at is CS Rankings, which ranks computer science programs based on publications in some of the top conferences in computer science.  In addition to the raw rankings (which can be highly misleading), the website also provides pie charts that give the breakdown of the publications by field.

For example, UC Berkeley is ranked top of the UC system and the pie chart shows particular strength in robotics, vision, and machine learning.  UC Santa Barbara is ranked as number 26, with particular strengths in security, databases, and electronic design automation (EDA).

But the particular method used is problematic, because of the reliance on a small number of conferences.  For example, UCSC’s strength in bioinformatics is not visible, because most of the publications are in biology journals (and glam journals like Nature), not in the two conferences that the rankings use (RECOMB and ISMB).  Conference publications are the main coin of the realm in most of CS, but not in bioinformatics, where journal publications rule. UCSC’s strength in storage systems is also not well represented, because FAST and USENIX ATC (two of the major conferences in that field) are not included.

The CSRankings.org method has some weird artifacts (like rewarding short author lists that exclude students and postdocs) and can be gamed in various ways.  Indeed, the ranking of institutions has changed enormously in the last two weeks, as departments have scrambled to up their averages by including affiliated faculty with high weighted publication counts and exclude those with low ones.

Despite the rather serious limitations of the CSrankings.org method, it is a more informative and useful system for comparing grad schools than ones like US News and World Report’s, which seem to bear almost no relationship with reality, being based almost entirely on hearsay reputation.

The best thing to use the CSRankings for is to look for what fields an institution is strong in (keeping in mind that many fields are not properly represented by the selection of conferences) and what faculty are publishing highly in the conferences that the CSRankings organizers think are important.  Grad school applicants should follow up by looking at the web pages of the faculty in the subfields that interest them (and check to make sure that the faculty are still there—people move around and a strong subfield 5 years ago may be missing now as faculty moved to industry or other academic positions).

A long PhD is not a bad thing

In response to http://xykademiqz.wordpress.com/2014/03/25/the-7-year-phd-itch, where she argued in favor of 5-year PhDs, and producing many papers as a grad student, I commented

I spent 8 years on my PhD (of course, I changed fields from pure math to computer science to computer engineering in that time). I only had a few papers when I was done, but I was in a hot new field and got a tenure-track position immediately. Unfortunately, it was not a good fit, and I ended up moving to another institution after 4 years, where it took me 7 more years to get tenure. So my BS-to-tenure time was 19 years. (The second job was a good fit, and I’m still at that university, though in a different field and in a different department.)

I find it difficult to advise students to race through grad school or to write huge numbers of crappy papers. I think that it is more important for students (and researchers in general) to write one or two high-quality papers that might actually make a difference.

Of the papers I wrote in grad school, one has never been cited (probably only one other person ever read it), one is my 6th most-cited paper (350 citations in Google Scholar and 86,600 hits with Google), and one has had very modest citations (85). My thesis itself was one-year throwaway work (only cited 9 times).

Note: I had fellowships for most of grad school, so only worked as an RA for 2 quarters and a TA for one. The highly cited paper was one that was not the result of any funded project, but an idea that another fellowship student came up with on his homemade computer and that we played with for a few years. The idea made over $100,000 in license fees for the campus and is what got me into the hot field that I was later hired for. I think that a lot has been lost by pushing students to be “hands in the lab” for senior researchers.

I’ve been sitting on this comment since March, with the idea of turning it into a full blog post.  I’ve seen a lot of different attitudes on the part of both grad students and faculty about how long a PhD should take and how much should be done for it.

My personal take is that a PhD education should be both broad and deep—one should have enough breadth of knowledge to teach several different undergrad courses and enough depth in one subject to have contributed original work to the field.

Research faculty generally want students to stick around for a fairly long time, so that they get payback in terms of co-authored papers for investment they have made (usually with Federal money) in the students’ initial training. A lot of them see no value to breadth, though, and just want someone to do the tough work in their lab.  They want students to start in research labs right away and see any time spent in coursework as wasted. These faculty often value research much more highly than teaching, doing the bare minimum teaching that the university lets them get away with—they also don’t pursue further education themselves, not attending any research seminars unless the seminar topics are directly tied to their current research projects.  The students they turn out are often very narrow researchers—good in one field, but not adaptable to changes in technology or research funding fads. Although these faculty often have impressive research teams, I’m not impressed with them as professors, as they have too narrow a view of what the role entails—they should be working in a private or national research lab rather than as professors at a university.

A more balanced professorial view sees the role of grad students primarily as students, learning how to be researchers and teachers, rather than as hired hands in the research lab.  As students, they should be continually learning new things, not just getting lab results in a narrow specialty.

Some grad students want to get the PhD certification as quickly as possible with as little effort as possible.  They generally end up in jobs that don’t require a PhD, so I don’t know why they bother—they’d be better off in most cases getting an MS degree (which is much faster) and going to work in industry.

Other grad students end up getting in a rut: not making much progress on their research, not taking any classes, not working on other research projects—basically just marking time.

Others start many projects, but don’t bring any of them to the state of completion needed for a thesis (that was me as a grad student—always busy, always learning, but not wrapping things up). Both the students in a rut and the students flitting from project to project may need to have their funding cut off, to motivate them either to finish theses quickly or give up—my thesis was written in a year after I was told I had only one year of funding left.  I think that there is some benefit to letting productive students have a free rein for a while, though—forcing students into a narrow niche too soon results in narrow researchers.

Some students try to turn their PhD thesis into a life work—as if the thesis is the best thing they’ll ever do.  This is a serious mistake that results in their staying a grad student for much too long. The point of a PhD thesis is to get the student a PhD—it is to establish that the student is capable of original work that contributes to the field and of writing that work up, no more. My own thesis was basically a throw-away research product.  By the time I was done with it, I realized that it was the wrong approach for tackling the design problem.  The only interesting part was a cute NP-completeness proof for a routing problem, all in pictures, but that was a time when new NP-completeness results were basically unpublishable, so I never bothered publishing it anywhere other than my thesis.

Having students do original work is not enough—the check that students can write things up is an important one. I’ve seen more students fail to get PhDs because they couldn’t write up their work than because they couldn’t do the research—that is one reason why our advancement to candidacy requirement consists mostly of writing a long, detailed research proposal, essentially a first draft of the thesis.  Students who can’t write either need to get help or find a job that does not require as much writing as most jobs that require PhDs.  (Incidentally, the problem of writer’s block often hits hardest those students whose writing is the best, when they can get it out—the problem is often one of perfectionism. So the strategy for addressing the problem has to be primarily psychological, not just instruction in writing.)

In recent years there has been considerable pressure on universities to pump students through faster, at both the undergraduate and graduate level. The effect has often been to deny students the chance to explore things outside a very narrow field—once undergrads have completed major requirements and university-mandated general education, there is no time left for other interests (and general-education requirements rarely are satisfied by other interests—they are usually mandated to be a bunch of low-level courses distributed across the curriculum to ensure butts in seats for various departments). Grad school pressure to reduce time-to-degree has often resulted in reducing the coursework requirements and getting students into research labs sooner, again reducing the breadth of student education.

Personally, I like “honors” programs, where at least the top students get released from the rigid bureaucratic requirements of general education and are free to shape idiosyncratic programs that get breadth and depth by following multiple interests, rather than by taking large numbers of survey courses.  I had such a program as an undergrad (the Honors College at Michigan State) and my son is currently in such a program (the College of Creative Studies at UCSB). It may not work for all students, but it is a good way to handle the students who are actually interested in learning things, not just in getting a degree.

In addition to my math degree, as an undergraduate I took a variety of other courses, some of which were interesting, some of which turned out to be duds. As a grad student, I continued this practice, and some of the just-for-fun courses turned out to be crucial to my future success.  For example, the computer music class lead to my taking the VLSI design class, in order to make a single-chip implementation of the plucked-string algorithm that Alex Strong and I had developed.  I ended up teaching VLSI design for over a decade, and the plucked-string paper is my 6th most-cited paper (365 citations on Google Scholar). Neither the plucked-string algorithm nor the VLSI design would have happened if Alex and I had followed the more conventional route of joining a professor’s lab and working on the problems that professor was funded for.  I would have finished my degree sooner, but would have developed a much narrower view of what research is worthwhile.  Although I took a long time as a grad student and a long time as an assistant professor, I still made tenure when I was 38, which is (just barely) below the average age for scientists getting tenure (over 39 according to Physics Today).

My son plans currently to take a lot of courses in his major (computer science), in his other academic interests (math, maybe physics and linguistics, maybe computer engineering), and in his recreational interests (acting)—it looks like he’ll only be required to take one or two classes that are of no interest to him.  He has taken more time in his pre-college schooling than I did, so he’ll probably not get his BS until he is 22 (I finished mine at 19), but he probably won’t need as long in grad school as me, because he’ll have had more time and opportunity to explore his interests earlier. (I certainly wasn’t ready to found a company at age 18!) For that matter, he might decide to go into full-time engineering with just a BS, and not go the academic route at all—his entrepreneurial spirit is more like his uncle than like his father.

Perhaps he’ll do what a lot of the students I teach have done: work for several years (or decades) in industry, then come back to grad school when bored with that, wanting a more interesting challenge.  The re-entry grad students generally do not take a long time to the PhD, because they are focused on their research, though they don’t seem to be much better than other grad students on planning what comes after the PhD.

My Laser Boyfriend’s Grad School Survival Guide

I’ve been meaning to recommend to all the grad students in my department that they read My Laser Boyfriend’s Grad School Survival Guide: How to make the most of the worst four to seven years of your life.

The advice in that blog post is excellent, particularly the parts about making friends with other grad students, finding a lab mentor, being nice to the lab techs, budgeting your time, backing up your computer, fixing things that need fixing, and seeking professional counseling if you get depressed.

Depression is a problem for grad students. I think that about 10% of grad students seek counseling each year, but that is based on a fairly small sample of one computational STEM department—it might vary with the field, with the mentoring styles of the faculty, and with the quality of the social support among the grad students. According to a 2008 blog post from PsychCentral, at UC Berkeley

within one year:

• About 45 percent experienced “an emotional or stress-related problem that significantly affected their well being and/or academic performance.”

• 10 percent “seriously considered suicide.”

They also reported

The majority of grad students actually don’t get help.

While some students consider seeking services, they don’t pursue them. For instance, the Berkeley study found that although nearly 52 percent thought about seeking services, only 27 percent did.

Also, international students are less likely to seek mental health services because of lack of knowledge about services and the stigma associated with mental illness and seeking help.

Their citation for the Berkeley Graduate Student Mental Health Survey has a broken link, but I think I found the 2004 survey as Appendix E to an information packet for the UC Regents, and I’ve provided a link to that packet.  The report does describe their survey method, which relied on responses to to an online survey, a sampling method that is likely to bias the results fairly strongly (but they got a 34.5% response rate, so the numbers can’t be more than a factor of 3 too large, even with the worst bias possible).

So it looks like my view of graduate student mental health may be particularly rosy. I don’t have access to confidential student medical records (nor should I), so it may be that there are more students seeking help in my department than I’m aware of.  Or it may be that our department is less stressful than many, or that our students are more competent than many and so suffer less from imposter syndrome (we do get to be very selective), or that our informal support mechanisms (like an awesome staff grad adviser) work better than most.  I think that my work as grad director for our department could be improved if I knew more about the mental health of the grad students in general, but I don’t know how much I can really do to improve conditions for them.

I’ve been putting off recommending My Laser Boyfriend’s blog post for one reason—the bad title.  There is no way that grad school should be the worst years of your life.  Yes, it is probably the most intense time of learning you will encounter, and you will finally meet and work with people smarter than you, possibly much smarter, but those are good things, not bad things.

Of course, I’m basing part of my opinion here on my own experience, where my 8 years of grad school were some of my best years—low stress, lots of cool things to do, and time to do them.  Having had fellowships that allowed me not to worry about funding or having to please a particular adviser who held the purse strings may have made a big difference here.  That’s one reason I encourage all grad students to apply for any fellowship that they might qualify for—it’s not just to stretch the funding budget but to give the grad students the freedom to explore topics that are not grant funded.

For those seeking an academic career, grad school should be a joy.  After all, the work load is lighter than that of an assistant professor, the schedule more flexible, and the stakes lower.  A student who finds grad school too stressful (under ordinary conditions, not counting demon advisers, family tragedies, or funding disasters) will probably find life as a professor miserable, and should seek a less-stressful work environment.

I’ve been told that the national labs and industrial research both provide less stressful environments, except for the fear of corporate mergers and massive layoffs in industrial jobs. My Dad worked at IITRI and at Argonne National Lab, and I do know that he was sometimes quite worried about the periodic layoffs, but he did not talk about it—so I don’t know how stressful those concerns really were. My own experience has all been in academia, other than one summer in Argonne National Lab and one summer in Bell Labs, both of which were student intern positions and not representative of full-time employment, so I don’t really know whether the national labs or industry are really less stressful than academia, or just different.  A fair comparison would probably require surveying hundreds of people who have had several years in both sorts of jobs and can compare them directly—I’m not interested in doing that sort of work, but I’d be glad of a pointer to research someone else has done.

Popular wisdom claims that the stress of an academic job drops enormously after getting tenure, since job security is then quite high.  I’ve not found that to be the case—it seems to me that the associate and full professors are just as stressed as the assistant professors, but more about how they will handle all the demands on their time and how they will fund their grad students and postdocs than by whether they will have a job in 5 years.

Thoughts for revising a grad curriculum, part 1

Our department is thinking of doing a major overhaul of our graduate curriculum. I’m going to do a series of posts on my thoughts about curricular design, both to clarify my own thinking and to provide a place where people can argue with me, so that I can either revise my ideas or come up with better justifications and explanations for them.

This first post is going to be fairly general, talking about broad goals that could apply to many different fields.  In later posts I hope to get down to nitty-gritty details that are more specific to bioinformatics and biomolecular engineering, and even to just our department.

We do little tweaks to our program every couple of years, but we haven’t done a really major revision since we set up the program in 2003.  Our department has tripled in size since then (we now have about 10 faculty), and each new faculty member has created at least one new graduate course, and so we now have more grad courses than our students can take.  Because we require our graduate students to take several courses from outside the department and have had a fairly steady number of grad students, we’ve ended up with too few students per course. With the recent and upcoming budget cuts, we can no longer afford the luxury of so many “boutique” grad courses.  Grad courses don’t need to be huge, but we can’t afford to have professors teaching classes with just 2 or 3 students unless they do it as overload (as I did last year).  We can’t build a grad curriculum under the assumption that the professors will do overload forever.

So one goal of the redesign is to reduce the number of grad courses offered each year, without depriving students of needed material.  We made a first stab at this already, by switching a lot of courses to alternate years (and cancelling a few completely) based on how popular the classes have been recently and how essential we thought the classes were. But this rationing of courses is not really a redesign—we can (and must) do better than that.

Students have also complained of some duplication of content between courses, though we have not yet done a careful analysis to find out what duplication they are seeing (there was little, if any, deliberate duplication, but faculty are not always aware of what other faculty have covered).  Obviously, de-duplicating material will allow us to reduce the number of courses without removing material, but I suspect that the duplication in the entire 9-course program amounts to substantially less than course, so it won’t help much in increasing efficiency.  It’ll make the students happy that they have been listened to, but would only result in minor edits, where a more major restructuring is needed.

A bigger problem to address is that our notion of the core of our discipline has evolved over the past decade, and so some of the requirements we set up at the beginning no longer seem to be a good fit.  As our department has grown, we’ve picked up new topics of research that call for rather different preparation than our original topics, and some of the topics we started with are no longer active areas of research. Ideally, we’d like to set up a system in which the topics can evolve more freely without having to redesign the requirements every five years.

Our faculty also have a variety of different models for what a graduate education is supposed to be, and these different models are sometimes incompatible.  Two extreme positions (which may not be held in pure form by anyone in the department) are the research-only model and the course-only model.  In the research-only model, students take no courses whatsoever and do nothing but work on one thesis project.  Producing an original piece of research is the sole goal of the research-only model, and anything that distracts from that goal is a waste of time.  In the courses-only model, graduate school is a continuation of undergraduate education, in which students take more specialized courses, eventually learning the most cutting-edge techniques of the field. The research-only model celebrates depth of learning, often at the expense of breadth, while the courses-only model leans in the opposite direction.

Because I characterized those as extreme positions, you can conclude that I don’t believe in either of them.  The model I have for graduate education is that it is a transition from learning about the field to becoming a professional researcher or professor in the field.  The requirements of a grad program need to support that transition, not assume that it has already been made nor that it can be put off into the indefinite future.

Because our graduates will go on to a variety of different careers (including industrial research, national labs, professorships at research universities, professorships at colleges that emphasize teaching, core bioinformatics support positions, software development positions, and others jobs that may not even exist currently), we need to prepare them for a broad range of possibilities. This argues against the research-only model, which prepares students only for pure research in the narrow specialty of their thesis—something almost none of our students end up doing as a career.

But we can’t be so broad and general that they come out being all “potential” with no track record of accomplishments. Even our MS program, which prepares students more for research support and software development positions than for research positions, must include projects that require substantial creativity and original work.

Currently our undergrad bioinformatics program relies heavily on first-year grad courses for the senior-year requirement.  The efficiency of teaching both the undergrads and grads in the same course is essential, particularly given that our undergrad program is small and unlikely to get much larger.

So here are some broad goals I have for the redesign:

• We should have an MS program that prepares students for research support and development careers.
• We should have a PhD program that prepares students for both research and teaching positions.
• The programs should be a suitable mix of breadth and depth, possibly balanced somewhat differently for the PhD and MS programs.
• Students should be able to transition fairly easily between the programs, as their career goals get more refined and their strengths and interests become more apparent.
• The grad courses should be teachable by our current faculty, even if 1/9th of them are on sabbatical.
• The requirements should be flexible enough to accommodate changes in the field, in the interests of the faculty, and in the backgrounds of our students, without needing frequent redesign of the curriculum.
• The grad program should provide needed courses for the senior year of our undergrad program.
• We should be able to teach all our courses, undergrad and grad, without faculty having to take on overload.

Undoubtedly I’ll think of more goals as soon as I post this, but I’ll save them for subsequent posts.

Fellowship applications

Every year at about this time I have my senior and first-year grad students write fellowship applications as an assignment for 2 classes (the bioinformatics core course and the how-to-be-a-grad-student course—most of the first-year students are in both, so can submit the assignment only once).

Because I’m on sabbatical this year, the courses are being taught by different people, so the students will get feedback from 3 people instead of one (a postdoc and TA who are teaching the core course, and a faculty member who is teaching the how-to-be-a-grad-student course).  Not satisfied with this, one of the new grad students organized an additional help session to get feedback from still more people on NSF fellowship applications (including students who had previously gotten NSF fellowships).  Feeling a bit left out of all this frenzy of fellowship feedback, I agreed to read and comment on three of the applications (less than the usual load of around 20 when I’m teaching the courses and way less than the grueling marathon of being on the NSF panel for grad fellowships, which I’ve done a few times in the past, but stopped doing because of how much time it ate out of my teaching and research).

The hope, of course, is that with all this polishing of the applications, more of the students will actually get fellowships (we’ve been averaging fewer than one a year, and our students are of a caliber where we should be getting 2 or 3 a year).

Mostly I’m looking for vague generalities (which trigger then panel’s BS detectors) and sloppy writing—those things are relatively easy to find and correct, but make a huge difference in the impact of the essay, particularly after 2 days of reading badly written essays.

I’ve read one and a half applications so far, and I can see that NSF would not fund them as written.  It is not that they are bad applications, but that they mention some no-no words for NSF: “biomedicine” and “drug interactions”.  NSF is very careful with their rather limited money, and won’t fund things that they think fall within the purview of the much wealthier NIH—anything having to do with medicine is out of scope for NSF.

Of course, grad students investigating those subjects are kind of stuck in a catch-22: NSF won’t fund fellowships for them, and NIH won’t fund grad students directly.  To get funding, they’d have to convince a faculty member to write a grant to support them, but the usual funding cycle for NIH grants means that no money would come in until far too late for the student (it probably averages more than 3 years from idea to getting any money from NIH, if you ever get that lucky).  NIH doesn’t want to pay for anyone except postdocs, who work very hard for very little money (a postdoc on a grant actually costs less than a grad student, since postdocs don’t pay tuition).

So what do I recommend to the students who have exciting work they want to do—work that does not fit under any existing grants of faculty in our department?  They can’t get money from NIH in a reasonable time frame, so NSF fellowships are still their highest-probability funding source.  (The NSF fellowships are too short, of course, at only 3 years, so they still have to write an NIH grant application with a faculty member, but at least the NSF fellowship is fairly quick turnaround (less than a year), so that they can have some money to do their research while waiting for the glacial grant cycles.

Those students who are interested in drug discovery or drug interactions, for example, have to spin their work differently.  Instead of focusing on the medical application (which in some cases was a bit dubious anyway), they should focus on biological discoveries and methods for discovering new ligand-protein interactions. Those interested in studying cell heterogeneity in cell lines by developing single-cell high-throughput experimental techniques have less rewriting to do—just cutting out mention of “medicine” may be enough.  It might also help if the work were done in something other than human cell lines, as human cell culture might trigger the knee-jerk “throw it over the fence to NIH” reaction.

The questions these students want to study are ones of fundamental science that can be applied to any species, not just to human medicine.  As such, they fall comfortably within NSF’s scope. The trick here is to keep the reviewers excited about the scientific possibilities, without triggering the NIH-should-pay-for-this reflex.

Note that this advice (to stay away from any mention of medical application and stick to fundamental biology) is precisely the opposite of the advice I would give to someone applying for an NIH grant, because NIH hates basic science and wants short-term application to medicine—the sooner the better. Although NSF is getting political pressure to become more applied, they are still more interested in fundamental science than in short-term payoff the way other funding agencies are.

Although, as an engineering faculty member I stand to gain from NSF becoming more applied, I do hope that NSF resists the politicians, as it is about the last source of funding for pure science research in the US.  Most highly applied “research” (properly called “development”) should be paid for more directly by industry (as much of it is), and the government should be funding more long-term science and engineering research.