Gas station without pumps

2015 November 11

Not applying for that grant after all

Filed under: Uncategorized — gasstationwithoutpumps @ 20:56
Tags: , ,

As long-time readers of my blog may know, I’ve given up on chasing grants (see Sabbatical Plans 2 and Sabbatical Leave Report), but I got sucked into planning to apply for NSF Engineering Education Program and Improving Undergraduate STEM Education/Professional Formation of Engineers’ RED Solicitation NSF 15-607, which would provide a minimum of $1,000,000 spread over 5 years to the lucky winner of the grant lottery for improving engineering education.  Because I have refocused my effort since my last sabbatical on improving education, this grant seemed like something worth some effort.

I was a little worried about it not being a lottery, but having an already targeted program that someone at NSF wanted to fund, as it had a very short timeline for putting together a rather complex grant, and somewhat bizarre requirements for the composition of the group applying for it:

The Principal Investigator(s) must be a department chair/head (or equivalent) to establish institutional accountability. Additionally, there must be a RED team that includes (at a minimum) an expert in engineering education or computer science education research, who can ground the research plan in the literature, and a social science expert who can evaluate department dynamics and monitor change processes. The social scientist must have expertise to advise on strategies for developing a culture of change and on strategies for creating meaningful collective ownership of the effort among faculty, students, and staff.

I was first informed of the existence of this program on 2015 Oct 7, by the engineering associate dean for undergraduates.  Apparently the deans of engineering schools had been informed of the program on 2015 Oct 5 by NSF, with letters of intent due on 2015 Nov 10, with each institution limited to 2 proposals.  I responded with cautious enthusiasm within an hour and a half, outlining what I’d like to see improved in the engineering program generally and why I thought that our Hispanic-Serving Institution was a good fit for the goals of the program to “educate inclusive communities of engineering and computer science students prepared to solve 21st-century challenges.”

I was willing to help write the grant, but I did not want to be the PI—not that I could anyway, as I’m a “Program Chair” but not a “Department Chair”—that means that I have to do all the catalog editing, curriculum revision, and responding to the administration about every bone-headed idea they come up with for education, but I have no resources and no carrots or sticks to get any other faculty to help me.

In my message to the faculty expressing interest, I detailed what I saw as the problems to address in the bioengineering program, some of which I felt were shared by other programs.

Another engineering faculty member (in a different department from mine) was in agreement with me, particularly on one point: “Students spend too much time getting book learning, and not enough time applying their knowledge to design problems.”  Our engineering programs have excellent senior capstone courses, but there is not enough design work in the first two years.  (Incidentally, this resonates well with a post that just came out today from a community college on the other side of the country.)

So within 2 hours of the associate dean asking if anyone was interested, the two of us agreed to work on it and see what we could come up with.  We both have heavy teaching loads this quarter, and he was working on several research proposals, so we did not manage to get together to talk for another nine days (Oct 16). We’d both done a fair amount of thinking independently before then, so we had a very productive meeting for an hour or two, finding that we had very similar ideas about the goals and complementary ideas about how to achieve them.

I got a couple of pages of notes out of that meeting: which courses needed to be expanded, which freshman and sophomore courses could feasibly have a greater design component, and how we could create and push courses back into the high schools to raise awareness of engineering among applicants (the other faculty member had already taught and recorded a summer course on robotics for high-school students that could be improved and adapted to be a “course-in-a-box” that could be taught by interested but not expert high-school teachers, and I would like to push my applied electronics course down to advanced high school level, though that would require some massive book rewrites).

The basic theme of our ideas was pretty straightforward (quoting from my notes on the Friday meeting):

The theme of the proposal is expanding hands-on project-based learning particularly in the majors Robotics Engineering, Computer Engineering, and Bioengineering (bioelectronics and assistive technology:motor concentrations).  Project-based learning has a good track record for increasing participation by women and under-represented minorities [citation needed].
The key concepts for the course and curriculum design are the following
  • System thinking: breaking into subproblems and well-defined interfaces
  • Trade-offs: most design decisions involve trading off one desirable feature for another
  • Documentation: the design needs to be thoroughly described in order to be maintainable or duplicable.

We concentrated on a part of the engineering program that already had a pretty good design component, trying to build from strength rather than trying to foment a revolution in programs that had very little design until the senior year.  Given the very short timeline (3.5 weeks to get a team together for the letter of intent), we did not think it wise to go for something unachievable, but rather to make a pretty good program exemplary.

Our next step was to see whether we could get a team together by the Nov 10 deadline for the letter of intent, so I started cold-calling (well, e-mailing) social scientists and education researchers on campus, trying to find people who would be suitable and interested. I’m not naturally a networker—I don’t remember people’s names or faces, and I don’t often go to social events where I run into new people, so I was having to rely on what I could find on the UCSC web pages and asking everyone for recommendations of whom to ask. I put in a fair amount of time looking through web pages and sending e-mail to strangers, asking for help.

Two weeks later (Oct 30), I managed to present the ideas of the proposal to a group consisting of one psychologist, three education researchers (one via a Skype connection that kept failing), and an EE teaching professor (who happened to be in the process of trying to improve the core EE course in the direction we were trying to move things).  The presentation must have seemed a bit bizarre to them, as it was the closest class day to Halloween, and I was dressed in a 15th-century houppelande, having just come from teaching my class.

After describing what we were trying to do and some lively discussion where the education researchers tried to figure out what NSF meant by their rather unusual team composition (not like any of the education research grants that they had ever participated in), I left with the EE professor eager to join the grant and the others saying they’d let me know.  By the next week, the psychologist (Nov 2) and the two best-fit education researchers (Nov 6) had agreed to join the team.

I had also had asked the dean’s office about the administrative support that had been promised in the original call for faculty interest, and got a rather minimal response (amounting to no more than the usual budget-writing support that tiniest research grants get—no grant writing support at all).

In the meantime (Nov. 5), another hurdle had arisen: the relevant department chair was not willing to be PI. Since we now had faculty from three different departments leading the grant, we tried convincing the dean to be the PI, but he’s stepping down at the end of the year, and did not feel that he could commit the incoming dean to whatever we were planning (Nov 9).  We made one more last-minute appeal to the department chair to let us file the letter of intent by the end of the day Nov 10, with the department chair still having veto power on submitting the final grant proposal, but were turned down.

So we’re not even getting a shot at the $1–2M lottery.  I suspect that many places that could have put together reasonable proposals will have had similar unsuccessful flurries of activity leading to not even being able to submit a letter of intent—the NSF request for proposals seemed deliberately structured to suppress applicants, leading me to suspect that there was a favored program somewhere that this whole charade had been set up to fund, or perhaps a few institutions with grant-writing machines already cranked up and ready to spew out whatever boilerplate NSF wanted.

The three of us faculty will go ahead and do what we can (without resources) to improve pedagogy in the engineering school, but the whole process has left a bit of a sour taste in my mouth. I’m feeling that not only did NSF not want proposals from us, but that the engineering administration didn’t want us applying for funding (which seems completely out of character for this university’s administration).

I think it is unlikely that I’ll go through that much effort again, just to be told that we can’t even file a letter of intent.  I’ve always hated grant writing, and I’d sworn off of research-grant writing a couple years ago as a completely unproductive use of my time.  Now it looks like I might swear off writing grant proposals for improving teaching also, as it seems to be even more painful and even less productive.

I would have been better off putting in the time revising another chapter of my book—at least there I can see progress when I can the time to work on it.

2014 January 25

NSF Idea Labs in STEM Education

Filed under: Uncategorized — gasstationwithoutpumps @ 15:27
Tags: , , ,

Mark Guzdial recently copied a NSF announcement about a new program that sounded interesting to me: NSF Dear Colleague Letter on new Idea Labs in STEM Education:

DEAR COLLEAGUE LETTER

Preparing Applications to Participate in Phase I Ideas Labs on Undergraduate STEM Education

NSF 14-033

The Directorate for Education and Human Resources has implemented a new program for “Improving Undergraduate STEM Education” (IUSE) through its Division of Undergraduate Education (EHR/DUE). The IUSE program description [PD 14-7513] outlines a broad funding opportunity to support projects that address immediate challenges and opportunities facing undergraduate science, technology, engineering, and math (STEM) education, as well as those that anticipate new structures and function of the undergraduate STEM learning and teaching enterprise. The IUSE program description creates an opportunity to submit unsolicited proposals across all topics and fields affecting undergraduate STEM education. It also includes an opportunity to participate in the first phase of three different Ideas Labs aimed at incubating innovative approaches for advancing undergraduate STEM education in three disciplines (biology, engineering, and the geosciences). These “IUSE Phase I Ideas Labs” will bring together relevant disciplinary and education research expertise to produce research agendas that address discipline-specific workforce development needs. The purpose of this Dear Colleague Letter is to provide additional information regarding the focus of the three Phase I Ideas Labs and guidance on preparing applications for community members seeking to participate in them.

There were two of the “Ideas Labs” that sounded relevant to me in my role as undergrad director and curriculum designer for the bioengineering program and as a bioinformatics teacher and researcher:

Biology

The biological sciences workforce for the future, including graduates of two-year schools, four-year institutions, and graduate programs, will need mathematical and computational skills beyond those of its predecessors. These tools also are required across the wide spectrum of biological sub-disciplines. …

Engineering

Social inequality in engineering education and practice is a durable problem, one that has resisted perennial efforts to “broaden participation,” “increase diversity,” or “improve recruitment and retention of women, minorities, and people with disabilities.” While a great deal of previous and ongoing work has focused on fostering the ability of individuals to access and persist in the engineering education system, this Ideas Lab will focus on changing the system itself. …

A five-day workshop with other people struggling with these problems might be interesting (though I don’t know how I could take a week off from teaching for an unknown week: one of March 3-7; March 17-21; March 31-April 4—they haven’t yet figured out which workshop will be which week).  Of those, only March 17–21 is at all feasible: missing the last day of class and exam week for this quarter. With a day of travel needed on each end, the ideas labs would take a full 7-day week.

I find writing proposals rather painful, and this one wouldn’t even result in any funding, so I re-read the letter more carefully to see if it really was something I wanted to do.

Page one of the “project description” seemed reasonable:

  • Provide a brief summary of your professional background (100 words maximum). Please note that if you are selected as a participant, information provided in answer to this question will be made available to the other participants, to facilitate networking at the Ideas Lab.
  • Describe your experience and interest in working across disciplines (100 words maximum).
  • Describe your key contribution(s) to addressing the specific STEM workforce development theme of this Ideas Lab (see above) through novel and potentially transformative approaches (no more than half a page).
  • Indicate your ability or inability to participate during any of the scheduled Ideas Lab dates (March 3-7; March 17-21; March 31-April 4).

But I found page 2 a bit difficult even to think about:

Please spend some time considering your answers to the following questions. Your responses should demonstrate that you have suitable skills and aptitude to participate in the Ideas Lab (unrelated to your research track record).

  • What is your approach to working in teams? (100 words maximum)
  • How would you describe your ability to engage non-experts or people with a different perspective to yours on this topic? (100 words maximum)
  • The Ideas Lab encourages a free exchange of ideas: enjoying the sharing, shaping and building ideas over an intensive 5-day setting, working as an equal with individuals you may not know.  How do you see yourself suited for this type of interaction?  If possible, describe any comparable experience you have had.  (150 words maximum).

I was also a little bothered by the description at the end of the engineering workshop:

In the Engineering Phase I Ideas Lab, engineers and social scientists will face head on the systems and structures that reproduce social inequality in engineering education and in the engineering workforce. A complete and direct discussion is not afraid to examine manifestations of racism, sexism, and ableism in engineering, and to also consider classism, heteronormativity, ageism, and obstacles faced by Veterans and other non-traditional groups. The Engineering Phase I Ideas Lab will generate new framings and new strategies to move the nation toward greater inclusion of marginalized groups in engineering.

In both places it seems clear that they have already decided precisely how to frame the question to get the answers they want to hear based on the jargon they use to describe the problem, and that they are not interested in hearing from any one who might disagree.  They seem to be trying to create a panel to rubber stamp some plan they have already devised, and are looking for a committee of yes-men to staff the panel.

I’m afraid I’m too much of a curmudgeon to be able to help them with that—I also have extreme doubts that they are going to succeed at anything with this plan but spending a lot of money on the pet ideas of the social scientists who came up with the plan, with no positive effect on engineering education or the “durable problem” of under-representation of women, blacks, Hispanics, and people with disabilities in engineering fields.

“Changing the system itself” sounds a lot like the MOOC advocates rallying call, as does the list of parameters they are proposing to modify:

Many prior efforts for inclusion have been hampered by a presumption that certain parameters can’t be changed (for example, eligibility criteria, narrow definitions of what counts in or as engineering, limited roles for 2-year institutions, or a four year degree model).

While I believe that ABET has been too narrow in some fields in their accreditation standards for engineering programs, I’m not convinced that throwing away what makes engineering a useful discipline is going to accomplish any socially useful goals. Applying the grade inflation and lowered expectations of other disciplines to engineering may indeed produce more diversity of graduates, but would probably make industry start insisting on higher qualifications, increasing the time and expense of meaningful engineering education, and shrinking rather than increasing diversity in the workforce.

The biology Ideas Lab has a more solvable problem:

The Biology Phase I Ideas Lab will consider strategies to integrate these critical competencies in quantitative literacy into a biology core curriculum and to study their effectiveness and/or impact to generate knowledge that will inform their broader implementation.

but I’m a little worried about the precise “quantitative literacies” that they have identified:

Specifically, these are “the ability to use quantitative reasoning” and “the ability to use modeling and simulation”, to gain a deeper understanding of the dynamics and complexity of biological systems. In addition, many areas of biology, from molecular, organismal through ecosystems studies, are reliant on large databases. Biologists of the future will require the mathematical and theoretical foundations necessary to abstract systems-level knowledge from complex data sets.  These skills will be important also for proper database management, preservation of the data collected, and effective use of the information they contain.

The emphasis on biologists needing to learn how to manage data and how to use large data sets is important, but the fact that they mention “mathematical” rather than “statistical” skills, and stress “modeling and simulation” implies to me that they are still stuck in the physicists’ mindset of differential equation modeling, which is not that compatible with the data biologists have available and the modeling that biologists need.  There is no mention of bioinformatics or statistics in this call, so I think that they are likely to be going in totally the wrong direction.

But their call for yes-men makes it clear that opinions like mine would not be welcomed—even if I could take off a week to sit around chewing the fat in Washington DC.  So I won’t be wasting my time trying to fill out the proposal forms by the Feb 4 deadline (the letter was only cleared for release on Jan 23, so it is clear they only want people who they had quietly hinted to ahead of time applying).

Oh well, politics as usual in Washington, DC.

2012 January 10

NSF “clarifies” Broader Impacts

Filed under: Uncategorized — gasstationwithoutpumps @ 21:08
Tags: , , ,

Since 1997 NSF has required that all grant proposals be evaluated on two dimensions: Intellectual Merit and Broader Impacts.

Intellectual Merit has always been clear to grant writers and grant reviewers (though they often disagree about the merits of any particular proposal).  “Broader Impacts” has always been rather murky with different grant writers and grant reviewers interpreting it in incompatible ways.  It has also been very difficult to evaluate whether “Broader Impacts” have been achieved by a funded project.

To try to clarify things, NSF just approved a report titled National Science Foundation’s Merit Review Criteria: Review and Revisions December 14, 2011, subject to final edits.  Unfortunately, they have not put the report on their own web site yet, so I had to get a copy from scienceinsider.  After reading the report (the body, not the hundreds of pages of appendices), I’m at least as confused as I was before about what the h*** NSF expects for Broader Impacts.

It seems that the guidance they offer is pretty much limited to one page:

Merit Review Criteria

When evaluating NSF proposals, reviewers should consider what the proposers want to do, why they want to do it, how they plan to do it, how they will know if they succeed, and what benefits would accrue if the project is successful. These issues apply both to the technical aspects of the proposal and the way in which the project may make broader contributions. To that end, reviewers are asked to evaluate all proposals against two criteria:

Intellectual Merit: The intellectual Merit criterion encompasses the potential to advance knowledge; and
Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

The following elements should be considered in the review for both criteria:

  1. What is the potential for the proposed activity to
    a. advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and
    b. benefit society or advance desired societal outcomes (Broader Impacts)?
  2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?
  3. Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?
  4. How well qualified is the individual, team, or institution to conduct the proposed activities?
  5. Are there adequate resources available to the PI (either at the home institution or through collaborations) to carry out the proposed activities?

They deliberately avoided further clarity:

Because of the great breadth and diversity of research and education activities that are supported by NSF, the Board has decided not to recommend a specific set of activities related to Broader Impacts, just as it would not recommend particular types of research–those decisions are best left to the PIs to describe and to the NSF to evaluate, for relevance to programmatic priorities and alignment with NSF’s core strategies for achieving its mission, as described in the NSF Strategic Plan for FY 2011- 2016 “Empowering the Nation Through Discovery and Innovation:”

  • Be a leader in envisioning the future of science and engineering.
  • Integrate research and education and build capacity.
  • Broaden participation in the science and engineering research and education enterprises.
  • Learn through assessment and evaluation of NSF programs, processes, and outcomes.

So grant writers and reviewers are left with nothing more specific than “benefit society or advance desired societal outcomes”. Different people will have very different ideas about what societal outcomes are “desired”, and Broader Impacts will continue to be used as a blunt instrument to reject intellectually worthy proposals based on the whims of reviewers.  If there were a list of “desired societal outcomes” then it might be possible to compare proposals, but without such a list, politics, religion, and random taste prevail.

More likely, there is a secret list of what NSF really wants for broader impacts, which is supposed to be “understood” without ever being stated.  So far as I can tell, NSF has just said “read our minds, we’re going to judge whether you address our goals without ever telling you what those goals are”. When this is combined with NSF’s new policy of funding bunches of stuff without external review, it becomes absolutely critical that there be clarity on what the goals are.

I hate guessing games like this.

2011 November 15

New NSF program without peer review

Filed under: Uncategorized — gasstationwithoutpumps @ 00:11
Tags: , , ,

In an attempt to be more nimble and interdisciplinary, NSF has instituted a new program that will consume up to 2% of NSF’s budget without peer review, just on a couple of program officers’ say-so. The official announcement, CREATIV: Creative Research Awards for Transformative Interdisciplinary Ventures, says that grants up to $1 million (spread over 5 years) can be funded this way.

The goals are to

  • Create new interdisciplinary opportunities that are not perceived to exist presently.
  • Attract unusually creative high-risk / high-reward interdisciplinary proposals.
  • Provide substantial funding, not limited to the exploratory stage of the pursuit of novel ideas.
  • Designate no favored topics; be open to all NSF-supported areas of science, engineering, and education research.

Furthermore

A CREATIV award must be substantially co-funded by at least two intellectually distinct NSF divisions or programs (this criterion is elaborated in the FAQ page. The maximum total award is $800,000 for two co-funding programs, and $1,000,000 for three or more co-funding programs. All awards are subject to the availability of funds.

and

Before writing and submitting a CREATIV proposal, it is the principal investigator’s responsibility to obtain written authorization to submit a CREATIV proposal by NSF program directors from at least two intellectually distinct divisions or programs.

This program looks like it may be accessible by bioinformatics researchers, but I fear that it removes some of NSF’s credibility, since it seems to fund things on a who-you-know basis, rather than on relying on peer review.  (Of course, ever since NSF copied NIH and started doing panel reviews rather than independent individual reviews of proposals, they have been sadly subject to groupthink, with the corresponding risk-averse selection of proposals.)

Hat-tip to ScienceInsider, whose article New NSF Program Sidesteps External Peer Review alerted me to this program.

2011 November 11

Fellowship applications

Every year at about this time I have my senior and first-year grad students write fellowship applications as an assignment for 2 classes (the bioinformatics core course and the how-to-be-a-grad-student course—most of the first-year students are in both, so can submit the assignment only once).

Because I’m on sabbatical this year, the courses are being taught by different people, so the students will get feedback from 3 people instead of one (a postdoc and TA who are teaching the core course, and a faculty member who is teaching the how-to-be-a-grad-student course).  Not satisfied with this, one of the new grad students organized an additional help session to get feedback from still more people on NSF fellowship applications (including students who had previously gotten NSF fellowships).  Feeling a bit left out of all this frenzy of fellowship feedback, I agreed to read and comment on three of the applications (less than the usual load of around 20 when I’m teaching the courses and way less than the grueling marathon of being on the NSF panel for grad fellowships, which I’ve done a few times in the past, but stopped doing because of how much time it ate out of my teaching and research).

The hope, of course, is that with all this polishing of the applications, more of the students will actually get fellowships (we’ve been averaging fewer than one a year, and our students are of a caliber where we should be getting 2 or 3 a year).

Mostly I’m looking for vague generalities (which trigger then panel’s BS detectors) and sloppy writing—those things are relatively easy to find and correct, but make a huge difference in the impact of the essay, particularly after 2 days of reading badly written essays.

I’ve read one and a half applications so far, and I can see that NSF would not fund them as written.  It is not that they are bad applications, but that they mention some no-no words for NSF: “biomedicine” and “drug interactions”.  NSF is very careful with their rather limited money, and won’t fund things that they think fall within the purview of the much wealthier NIH—anything having to do with medicine is out of scope for NSF.

Of course, grad students investigating those subjects are kind of stuck in a catch-22: NSF won’t fund fellowships for them, and NIH won’t fund grad students directly.  To get funding, they’d have to convince a faculty member to write a grant to support them, but the usual funding cycle for NIH grants means that no money would come in until far too late for the student (it probably averages more than 3 years from idea to getting any money from NIH, if you ever get that lucky).  NIH doesn’t want to pay for anyone except postdocs, who work very hard for very little money (a postdoc on a grant actually costs less than a grad student, since postdocs don’t pay tuition).

So what do I recommend to the students who have exciting work they want to do—work that does not fit under any existing grants of faculty in our department?  They can’t get money from NIH in a reasonable time frame, so NSF fellowships are still their highest-probability funding source.  (The NSF fellowships are too short, of course, at only 3 years, so they still have to write an NIH grant application with a faculty member, but at least the NSF fellowship is fairly quick turnaround (less than a year), so that they can have some money to do their research while waiting for the glacial grant cycles.

Those students who are interested in drug discovery or drug interactions, for example, have to spin their work differently.  Instead of focusing on the medical application (which in some cases was a bit dubious anyway), they should focus on biological discoveries and methods for discovering new ligand-protein interactions. Those interested in studying cell heterogeneity in cell lines by developing single-cell high-throughput experimental techniques have less rewriting to do—just cutting out mention of “medicine” may be enough.  It might also help if the work were done in something other than human cell lines, as human cell culture might trigger the knee-jerk “throw it over the fence to NIH” reaction.

The questions these students want to study are ones of fundamental science that can be applied to any species, not just to human medicine.  As such, they fall comfortably within NSF’s scope. The trick here is to keep the reviewers excited about the scientific possibilities, without triggering the NIH-should-pay-for-this reflex.

Note that this advice (to stay away from any mention of medical application and stick to fundamental biology) is precisely the opposite of the advice I would give to someone applying for an NIH grant, because NIH hates basic science and wants short-term application to medicine—the sooner the better. Although NSF is getting political pressure to become more applied, they are still more interested in fundamental science than in short-term payoff the way other funding agencies are.

Although, as an engineering faculty member I stand to gain from NSF becoming more applied, I do hope that NSF resists the politicians, as it is about the last source of funding for pure science research in the US.  Most highly applied “research” (properly called “development”) should be paid for more directly by industry (as much of it is), and the government should be funding more long-term science and engineering research.

%d bloggers like this: