# Gas station without pumps

## 2014 October 26

### Critical thinking

In a comment exchange on the Cost of College post Trying to teach the enigmatic and increasingly popular skill of critical thinking, CSProfMom and I were discussing the relationship between critical thinking and the various modes of thinking used in CS and engineering.  I’ll pull out some of the highlights of the discussion to set the stage for this post, but I’m leaving some things out, so go read the original post and comments.

Grace, the author of the post, presented some of the definitions of critical thinking she had found, and CSProfMom replied with

I do not like these definitions of critical thinking because they are only based on verbal reasoning. Mathematical and computational problem solving are utterly ignored; yet I think more critical thinking goes on in those areas than in fields like literary analysis.

Grace defended the definitions, and CSProfMom responded with CMU’s definition of computational thinking:

Computational thinking means creating and making use of different levels of abstraction, to understand and solve problems more effectively.

Computational thinking means thinking algorithmically and with the ability to apply mathematical concepts such as induction to develop more efficient, fair, and secure solutions.

Computational thinking means understanding the consequences of scale, not only for reasons of efficiency but also for economic and social reasons.

http://www.cs.cmu.edu/~CompThink/

I weighed in with

I think that CSProfMom’s point is that “critical thinking” is generally defined rather narrowly and generically, and so misses the important thinking styles that are crucial to some fields. “Computational thinking” is one that is missing. One I see students not getting in most of their college classes is “engineering thinking” or “systems thinking”—dividing difficult problems into simpler subproblems with clearly defined interactions between the subproblems. Although one can argue that these specific modes of thinking are somehow subsumed in “critical thinking”, classes that purport to develop critical thinking skills don’t generally develop these important modes of thinking.

CSProfMom responded with

I think there is a lot of overlap between “computational thinking”, “mathematical thinking”, and “systems thinking”. Abstraction and decomposition are key skills in all three. Your description “dividing difficult problems into simpler subproblems with clearly defined interactions” is absolutely critical in computer science. Maybe computational thinking is simply systems thinking + algorithms?

In any case, because the “critical thinking” term does not include this idea of systems thinking, we see students arrive into our engineering/CS programs utterly unable to think in this manner. I believe that is a major reason why we see the terrible attrition rates in these programs.

The rest of this post will be an expansion on the comment I left in response to this.

There are several different terms floating around in our discussion, and I’d like to pull them out for closer examination:

critical thinking
This seems to be a subset of the medieval trivium (grammar, logic, and rhetoric), leaving out the grammar and being a bit light on the rhetoric. It doesn’t even cover modern mathematical logic, but only the simplest Aristotelian logic. The Wikipedia article on the trivium even points to the critical thinking article, which collects nine conflicting definitions of critical thinking, none of which include the other concepts that I list below, except in the vaguest ways.
mathematical thinking
Mathematical thinking is about setting up formal systems of rules and examining their properties very closely. Proofs are a major component of mathematical thinking, which has a much more formal and unforgiving definition of proof than other fields. Computation has created a lot of new formal systems to study, and has been a fruitful area recently for mathematicians, just as physics was in previous centuries. Theoretical computer science is basically a branch of mathematics, involving primarily mathematical thinking.
scientific thinking
Scientific thinking studies the real world, constructing models of how it functions and testing the models empirically.  Different branches of science differ in how much they are model-driven and how much they are data-driven. Physics is highly model-driven, with the models often coming out 40 or 50 years in advance of the data (see Higgs boson).  Biology is highly data-driven often with non-quantitative  verbal stories as the models.  The key concept throughout science is empirical validation of predictive models.
engineering thinking
Engineering is about designing new things.  An engineering question is more of the form “how can I make this work?” rather than the science question “how does this work?”  I’ve talked about the distinction between science and engineering in one of my early blog posts, so I won’t belabor the point here.  Although scientists and engineers often wander back and forth between scientific and engineering thinking, the two are really distinctly different modes of thought.
systems thinking
Systems thinking is an important component of engineering thinking, consisting of dividing difficult problems into simpler subproblems with clearly defined interactions between the subproblems.  But systems thinking cuts across many fields, including mathematical thinking and scientific thinking.
Computer programming is one of the best subjects to teach systems thinking in, because computer languages provide formal (though still inadequate) ways of representing the modules that encapsulate the subproblems and the interactions between them.  Electrical engineers try to do the same thing with their block diagrams, but these formalize a little less of the interactions, relying on English-language descriptions that are often omitted or poorly written.
Unfortunately, many of the lower-level computer science classes have the faculty or textbook authors do all the systems thinking for the students, so that the students never learn to do it themselves. The scaffolding put in place to help the students find good solutions is where all the systems thinking happened, and descaffolding so that students have to learn to subdivide difficult problems into easier ones is an essential, but often missing, component of teaching programming.
The “multiple levels of abstraction” mentioned in the CMU definition of computational thinking is really about systems thinking, as each subproblem gets broken down into still smaller problems.
algorithmic thinking
Algorithmic thinking is coming up with very precise recipes for doing things—not just flailing around trying things, but having very specific methods that can be shown to work (and work efficiently). Algorithmic thinking is really a branch of mathematical thinking, interested in provably correct manipulations in formal rule systems.  Originally it was applied to computing numerical functions, first manually and later by machine, but now has been expanded to cover many different types of data that can be represented in computers.  This is the second part of the CMU definition of computational thinking.
computational thinking
I don’t like the CMU definition of computational thinking, as they seem to have picked up definitions of mathematical, systems, and algorithmic thinking, and missed the computational part entirely. Computational thinking, to me, involves using computation to solve problems (data analysis, algorithmic solution of symbolic problems, numerical simulation, …) and may not involve much systems thinking or algorithmic thinking—someone else may have done that for you to enable you to use a computational tool.  Using Google to search for information is computational thinking, albeit at a rather low level.
statistical thinking
Statistical thinking is distinct from all of the above, though it is often important in scientific thinking.  Statistical thinking involves reasoning about data that comes from random processes, or that can be modeled as having been corrupted by random noise.  Notions of probability, sample size, statistical significance, multiple-hypothesis correction, correlation, and causation are all part of statistical thinking, which has applications to decision making in all aspects of life.

Obviously, there are overlaps and intersections between these different modes of thought (proofs done with the aid of a computer are a combination of mathematical and computational thinking, for example), but there are important differences also.  For example, Engineering thinking is not just systems thinking, but includes attention to fine details in the edge conditions (a hallmark of mathematical thinking), making allowances for variations in manufacturing (statistical thinking), testing how the device works in the real world (scientific thinking), and, very often these days, figuring out how to use cheap microcontrollers to do tasks that traditionally were done with more expensive analog devices (computational thinking).

The UCSC general education requirements (see my blog post on general education) recognize mathematical reasoning, scientific inquiry, and statistical reasoning as distinct parts of general education, adding textual analysis and cross-cultural analysis to cover what is often lumped under “critical thinking”.  They did not include anything that guarantees exposure to systems thinking, and they tossed in a few other things, some of which seem to me to be more politically expedient choices or fashion following than fundamental modes of thinking, but a general education system is always a compromise between different beliefs about what a university education should mean.  I think they did a better job of defining the basis for their general education system than most universities manage.

There have been a lot of calls for more education in “critical thinking” lately.  I’m not really happy with these calls, because teaching only a weakened version of the medieval trivium instead of more powerful modern forms of thinking does not educate students properly.

## 2013 July 4

### Caballero on teaching physics with computation

Yesterday, Danny Caballero left a comment on my blog with pointers to three preprints of papers he’s been a co-author on:

I couldn’t agree more with the overall sentiment in this post. Computing is important for all students of science and for science teachers. However, as you say, I do think my dissertation work has been misconstrued a bit in this post. So, let me clarify.

The inventory you mention measures a small slice of mechanics taught in introductory physics. It’s been an important slice, but maybe now it’s time to think beyond it. What are students learning above and beyond this assessment? In M&I, they are learning about modeling, computing, and connecting the two. My work shows that the present implementation of M&I doesn’t produce great gains on this assessment, that students make mistakes in their code, and that they are less inclined towards computing after instruction.

So what? We are getting to teach students how science is done and they are using computing to investigate models. Now, this implementation is not the most polished one, which means we have a good way to go. But, that’s OK, because we are not going to fix all these issues overnight. We have been working on them for the last two years in various contexts. But, we need to figure out how to teach science and computing together. And we need to figure out how to do it well.

So, I’ll point you to a few other publications on computing in physics that I’ve written. Two concern high school, and another deals with physics majors. The high school work shows we can implement ideas from my dissertation work at the high school level (as others are doing). Moreover, we find that students who know physics and computing ideas can make good models of systems and are not memorizing lines of code. In my dissertation work, we didn’t do any qualitative work like student interviews, but it’s clear that doing so is necessary. The work with physics majors is one of the first forays into integrating computing in upper-division physics. We show that a new model for implementation can positively affect student attitudes.

Upper-division:
http://arxiv.org/abs/1303.4355

This morning I read his three papers.  They all describe prototype courses that use computational modeling to teach physics, with some analysis of the outcomes.  They are not controlled experiments, but prototyping proof-of-principle projects (as are most educational research “experiments”).  This is not a criticism of the papers—one generally has to do a lot of prototyping before arriving at a method that is robust and repeatable enough to be worth the difficulty and expense of controlled experiments.  Engineers see the need for prototyping, but too many people in other fields think that things have to work perfectly the first time or be discarded forever.

These papers bracket the previous work Danny did that studied computational modeling for first-year calculus-based physics.  The two high-school papers are about a 9th grade physics class that uses the Modeling Instruction approach to teaching physics, to which was added computational modeling using VPython.  The “upper-division” paper discusses adding computational modeling to a 2nd-year classical mechanics course for physics majors, following a traditional 1st-year calculus-based physics course.

I was a little unclear on the level of the 9th-grade course.  In one place he refers to it as “conceptual physics”, but in other parts of the description it sounds more like an algebra-based high school physics course (covering the mechanics half of AP Physics B), a step-up from conceptual physics.

From his description, it seemed fairly straightforward to add a computational component to the Modeling Instruction approach, and it helped students see that all the different “models” taught in that approach are really special cases of the same underlying general model.  They used Vpython with a couple of additional packages (PhysKit and PhysUtil) to make creating graphs and motion diagrams easier for beginning programmers.  The additional packages allow lines like

```    motionMap.update(t)
timerDisplay.update(t)
graph.plot(t, cart.pos.x)
```

in the inner loop, simplifying the usual VPython interface a bit.

It sounds like the students were finishing the course with a mix of students who knew what they were doing and those who still hadn’t quite grasped the physics or hadn’t quite got the programming.  He did try analyzing some of the student work to see whether students were having difficulty with the physics or VPython for making the simulations, but I found the results hard to interpret—raw numbers don’t mean much to me, because I don’t have a good prior expectation of what 9th graders at a private high school should be able to do.  I’m curious whether difficulties with programming correlated with difficulties in understanding the physics, and whether grading the computational homework gave insight into the misconceptions the students had about the physics.

One of the strong points of the computational approach is that it allowed the students to model phenomena usually beyond the scope of 9th-grade physics (like a soccer ball with linear drag forces).  I found this to be the case for calculus-based physics also, where we modeled pendulums without the small-angle approximation (problem 4.P.89 in Matter and Interactions) and the magnetic field lines of a helical solenoid.

Some of his observations are unsurprising: “Students find debugging their programs difficult; that is, they have trouble determining whether they have made a coding error or a physics error and how to deal with that issue. ”  He also noticed that some students found installing the software difficult—I believe that the VPython developers have been working on that problem, though it is not yet at the level where all 9th graders will find it easy.

Personally, I’d like to see the physics simulations for high school students use computations with units throughout—this would help them catch a lot of their physics errors earlier.  I see this lack of units as one of the biggest flaws in VPython as an instructional tool for physics.  I’ve blogged about this before in Physics with Units, and I’ve done some VPython programming using Unum.  Unfortunately, the Vpython plotting and animation code does not play nicely with Unum, and having to strip out the units before plotting or drawing negates most of the advantages of keeping units around. I realize that for professional physics simulations, units are always implicit (in comments and variable names) rather than explicit, because that makes more efficient use of the computer, but for instructional purposes explicit units would be worth the inefficiency.

The 2nd-year classical mechanics course used Mathematica to solve ordinary differential equations (ODEs), and provided only minimal instruction in Mathematica.  The main improvement to the course from my perspective was the addition of a final project that allowed students to study an open-ended physics question of their own choice using computational modeling.  This improvement was discarded in subsequent offerings, because it required too much instructor time. Caballero wrote, “For junior and research-focused faculty, the computational project is a significant investment of their time and energy given the large enrollment in CM 1. Developing authentic, scientific experiences for students that can be sustained with little faculty input is challenging.”

This is a theme that I see repeatedly in course design in all disciplines and at most universities: the really good parts of prototype courses take a lot of instructor time and get discarded.  I think that the goal “sustained with little faculty input” is a wrong goal, but it is one shared by many faculty and administrators, who think that teaching is a burden that should be given as little effort as they can get away with. I’ve decided, rather deliberately, not to design my courses that way, but to design them around high faculty involvement.  I believe that the value of a University education depends on high-contact courses, and I’m willing to resist the MOOCification of the university at least in my own courses.  This does take a lot of my time, and I’ve given up on writing grant proposals to make the time—not a choice that most junior faculty could afford to make, given that promotion these days is based more on how much money is brought in than on the quality of teaching or research.

## 2012 April 3

### Computer Science for Non-Majors

Filed under: Uncategorized — gasstationwithoutpumps @ 21:57
Tags: , , , ,

Mark Guzdial, in his Computing Education Blog (Weekend newspaper coverage on computing for all, especially kids), commented on the NY Times article by Randall Stross: Computer Science for Non-Majors Takes Many Forms.

The article makes the case that everyone should be taught “computational thinking”, though Stross does not attempt to define what that means, merely noting

There is little agreement within the field, however, about what exactly are the core elements of computational thinking. Nor is there agreement about how much programming students must do, if any, in order to understand it.

Stross gives examples from Carnegie-Mellon and from Wheaton College in Massachusetts of two rather different approaches—one using Ruby and doing fairly conventional CS things (like iteration, recursion, random number generators, … ) and the other tackling analysis of large bodies of text by humanities majors, starting from big problems and helping the students decompose them into smaller, more solvable ones.

Stross even mentions the course using Scratch at the University of Maryland, Baltimore County, though I believe that he is too dismissive of Scratch as a programming language.  Yes, it is easy enough for middle-schoolers to learn it, but that doesn’t mean that it can’t be used for teaching most of the concepts of a first-year computer course.  (OK, Scratch is not suitable for teaching recursion, but it is very suitable for teaching an introduction to parallel programming and race conditions, which most languages used in first-year CS classes are not.)  I think that Marie desJardins and Michael Littman (of Rutgers) gave weak replies—Littman saying that

Computational thinking should have been covered in middle school, and it isn’t. So we in the C.S. department must offer the equivalent of a remedial course.

While it is certainly true that many intro to CS classes are the equivalent of remedial classes in math, science, or writing, it is not the case that all intro to CS classes need to be remedial.

Mark Guzdial seemed a little miffed that he was interviewed but not quoted:

I exchanged email with the author before the column came out, pointing out several sources and mentioning Media Computation. Randy felt that Georgia Tech’s experience with computing for everyone was less compelling, because Georgia Tech is an “engineering school.”  He said that he was more interested in programs that cater to humanities majors — which is what MediaComp is, because we developed it to reach Liberal Arts, Architecture, and Management majors.

I guess I would have been miffed to have been so misunderstood by a business professor also.

Mark goes on to talk about somewhat better coverage of similar issues in The Guardian, which had a section focused on computing education. He particularly liked Naughton’s “manifesto for revolutionary action“. I too like point 3:

We believe every child should have the opportunity to learn computer science, from primary school up to and including further education. We teach elementary physics to every child, not primarily to train physicists but because each of them lives in a world governed by physical systems. In the same way, every child should learn some computer science from an early age because they live in a world in which computation is ubiquitous. A crucial minority will go on to become the engineers and entrepreneurs who drive the digital economy, so there is a complementary economic motivation for transforming the curriculum.

Naughton does recognize some problems with his vision:

We recognise that a key obstacle to achieving the necessary transformation of the computing curriculum is the shortage of skilled and enthusiastic teachers.

Although the manifesto makes some very specific calls for changes in the UK, I think that Naughton’s explanation in a separate column is more compelling reading, explaining why things need to be different.  In it he defines “computational thinking” in a fairly broad way:

… computer science involves a new way of thinking about problem-solving: it’s called computational thinking, and it’s about understanding the difference between human and artificial intelligence, as well as about thinking recursively, being alert to the need for prevention, detection and protection against risks, using abstraction and decomposition when tackling large tasks, and deploying heuristic reasoning, iteration and search to discover solutions to complex problems.

I don’t know that Naughton’s definition matches my idea of “computational thinking” exactly (the stuff about risks is part of computer science, but not computational thinking, in my opinion), but it is a better shot at the definition than some I’ve seen.  His examples are rather feeble though (computational biology is not about “how the genetic code works”).

Despite my nit-picking, it is a good weekend when two major newspapers independently decide to talk about computer science education, and what needs fixing in it.

## 2011 September 24

### On Theoretical/Computational Sciences

GMP (which is short for GeekMommyProf, not Gay Male Prof, as I initially thought), put out a call for blog entries in her blog: Academic Jungle: Call for Entries: Carnival on Theoretical/Computational Sciences, asking computational and theoretical scientists to write “a little bit about what your work entails, what you enjoy/dislike, what types of problems you tackle, what made you chose your specialization, etc.” (Deadline Sunday Sept 25)

As a bioinformatician, I certainly do computational work, but probably not in the sense that GMP means.  My computational work is not theoretical work, nor modeling physical processes using computational models.  Bioinformatics is not a model-driven field the way physics is.  (In fact, the very lumping together of theoretical and computational work implies to me that GMP is a physicist or chemist, though her blogger profile is deliberately vague about the field.)

To give an example of data-driven computational work that is not model-based, consider the example of predicting the secondary structure of a protein.  In the simplest form, we are trying to predict for each amino acid in a protein whether it is part of an α-helix, a β-strand, or neither.  The best prediction methods are not mechanistic models based on physics or chemistry (those have been terribly unsuccessful—not much better than chance performance).  Instead, machine learning methods based on neural nets or random forests are trained on thousands of proteins.  These classifiers get fairly high accuracy, without having anything in them remotely resembling an explanation for how they work. (Actually, almost all the different machine-learning methods have been applied to this classification problem, but neural nets and random forests have had the best performance, perhaps because those methods rely least on any prior understanding of the underlying phenomena.)

It is rare in bioinformatics that we get to build models that explain how things work.  Instead we rely on measuring the predictive accuracy of “black-box” predictors, where we can control the inputs and look at the outputs, but the workings inside are not accessible.

We judge the models based on what fraction of the answers they get right on real data, but we have to be very careful to keep training data and testing data disjoint. It is easy to get perfect accuracy on examples you’ve already seen, but that tells us nothing about how the method would perform on genuinely new examples.  Physicists routinely have such confidence in the correctness of their models that they set their parameters using all the data—something that would be regarded as tantamount to fraud in bioinformatics.

Biological data is not randomly sampled (the usual assumption made in almost all machine-learning theory).  Instead, we have huge sample biases due to biologists researching what interests the community or due to limitations of experimental techniques.  As an example of intensity of interest, of 15 million protein sequences in the “non-redundant” protein database, 490,000 (3%) are from HIV virus strains, though there are only a dozen HIV proteins.  As an example of limitations of experiments, there are fewer than 900 structures of membrane proteins, out of the 70,000 structures in PDB, even though membrane proteins make up 25–35% of proteins in most genomes.  Of those membrane protein structures, there are only about 300 distinct proteins (PDB has a  lot of “redundancy”—different structures of the same protein under slightly different conditions).

One of the clearest signs of a “newbie” paper in bioinformatics is insufficient attention to making sure that the cross-validation tests are clean.  It is necessary to remove duplicate and near-duplicate data, or at least ensure that the training data and the testing data have no close pairs.  Otherwise the results of the cross-validation experiment do not provide any information about how well the method would work on data that has not been seen before, which is the whole point of doing cross-validation testing.  Whenever I see a paper that gets astonishingly good results, the first thing I do is to check how they created their testing data—almost always I find that they have neglected to take the standard precautions against fooling themselves.

I came into bioinformatics through a rather indirect route: B.S. and M.S. in mathematics, then Ph.D. in computer science, then 10 years as a computer engineering professor, teaching VLSI design and doing logic minimization.  I left the field of logic minimization, because it was dominated by one research group, and all the papers in the field were edited and reviewed by members of that research group.  I thought I could compete on quality of research, but when they started rejecting my papers and later publishing parts of them under their own names, I knew I had to leave the field (which, thanks largely to their driving out other research groups, has made almost no progress in the 15 years since).  While I was looking for a new field, I happened to have an office next to a computer scientist who was just moving from machine learning to bioinformatics.  I joined him in that switch and we founded a new department.

I had to learn a lot of new material to work in my new field (Bayesian statistics, machine learning, basic biology, biochemistry, protein structure, … ).  I ended up taking courses (some undergrad, but mostly grad courses) in all those fields.  Unlike some other faculty trying to switch fields, I had no grant funding for time off to learn the new field, nor any reduction in my teaching load.  Since I have hit a dry spell in funding, I am on sabbatical this year, looking at switching fields again (still within bioinformatics, though).

If I were to do my training and career over again, I would not change much—I would switch fields out of logic minimization sooner perhaps (I hung on until I got tenure, despite the clear message that the dominant group felt that the field wasn’t big enough for competitors), and I would take more science classes as an undergrad (as math major, I didn’t need to take science, and I didn’t).  I would also take more engineering classes, both as an undergrad and grad student. I’d also take statistics much sooner (neither math nor computer science students routinely take statistics, and both should).  I’d also up the number of courses I took as a professor, to a steady one a year.  (I was doing that for a while, but for the past few years I’ve stopped, for no good reason.)

## 2011 March 27

### Computational Thinking

Filed under: Uncategorized — gasstationwithoutpumps @ 10:07
Tags: , ,

In Research Notebook: Computational Thinking—What and Why?, Jeannette M. Wing presents a definition of computational thinking and an explanation of why it is important for everyone, not just computer scientists.

Her definition is

Computational thinking is the thought processes involved in formulating problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent. [Jan Cuny, Larry Snyder, and Jeannette M. Wing, “Demystifying Computational Thinking for Non-Computer Scientists,” work in progress, 2010]

though she credits Al Aho for some of the thought behind it.

A key part of computational thinking is the value of levels of abstraction: being able to represent just the “relevant” information and manipulate it, without having to worry about lower and higher levels of representation, other than in explicit terms of the current level.  To my mind, good computational thinking requires being able to switch levels of abstraction and to work in 2 or more levels at once, but just being able to abstract problems and hide some of the less relevant details is a valuable skill.

Jeannette gives some of examples of computational thinking in everyday life, one of which particularly rang a bell with me:

Hashing: After giving a talk at a department meeting about computational thinking, Professor Danny Sleator told me about a hashing function his children use to store away Lego blocks at home. According to Danny, they hash on several different categories: rectangular thick blocks, other thick (non-rectangular) blocks, thins (of any shape), wedgies, axles, rivets and spacers, “fits on axle,” ball and socket and “miscellaneous.” They even have rules to classify pieces that could fit into more than one category. “Even though this is pretty crude, it saves about a factor of 10 when looking for a piece,” Danny says. Professor Avrim Blum overheard my conversation with Danny and chimed in “At our home, we use a different hash function.”

The analogy is not perfect, as there is usually no collision resolution method when a bin gets too full.  (In our house, collision resolution usually requires reallocation of space, which is a very expensive operation, just as in programming. Rehashing has been known to take months, and the temporary space used for the rehashing operation is not available for other tasks.)

Note: I’ve posted on computational thinking twice before: Algorithmic vs. Computational Thinking and Computational Thinking Lesson Plans.

Next Page »