Critical thinking

In a comment exchange on the Cost of College post Trying to teach the enigmatic and increasingly popular skill of critical thinking, CSProfMom and I were discussing the relationship between critical thinking and the various modes of thinking used in CS and engineering.  I’ll pull out some of the highlights of the discussion to set the stage for this post, but I’m leaving some things out, so go read the original post and comments.

Grace, the author of the post, presented some of the definitions of critical thinking she had found, and CSProfMom replied with

I do not like these definitions of critical thinking because they are only based on verbal reasoning. Mathematical and computational problem solving are utterly ignored; yet I think more critical thinking goes on in those areas than in fields like literary analysis.

Grace defended the definitions, and CSProfMom responded with CMU’s definition of computational thinking:

Computational thinking means creating and making use of different levels of abstraction, to understand and solve problems more effectively.

Computational thinking means thinking algorithmically and with the ability to apply mathematical concepts such as induction to develop more efficient, fair, and secure solutions.

Computational thinking means understanding the consequences of scale, not only for reasons of efficiency but also for economic and social reasons.

http://www.cs.cmu.edu/~CompThink/

I weighed in with

I think that CSProfMom’s point is that “critical thinking” is generally defined rather narrowly and generically, and so misses the important thinking styles that are crucial to some fields. “Computational thinking” is one that is missing. One I see students not getting in most of their college classes is “engineering thinking” or “systems thinking”—dividing difficult problems into simpler subproblems with clearly defined interactions between the subproblems. Although one can argue that these specific modes of thinking are somehow subsumed in “critical thinking”, classes that purport to develop critical thinking skills don’t generally develop these important modes of thinking.

CSProfMom responded with

I think there is a lot of overlap between “computational thinking”, “mathematical thinking”, and “systems thinking”. Abstraction and decomposition are key skills in all three. Your description “dividing difficult problems into simpler subproblems with clearly defined interactions” is absolutely critical in computer science. Maybe computational thinking is simply systems thinking + algorithms?

In any case, because the “critical thinking” term does not include this idea of systems thinking, we see students arrive into our engineering/CS programs utterly unable to think in this manner. I believe that is a major reason why we see the terrible attrition rates in these programs.

The rest of this post will be an expansion on the comment I left in response to this.

There are several different terms floating around in our discussion, and I’d like to pull them out for closer examination:

critical thinking

This seems to be a subset of the medieval trivium (grammar, logic, and rhetoric), leaving out the grammar and being a bit light on the rhetoric. It doesn’t even cover modern mathematical logic, but only the simplest Aristotelian logic. The Wikipedia article on the trivium even points to the critical thinking article, which collects nine conflicting definitions of critical thinking, none of which include the other concepts that I list below, except in the vaguest ways.

mathematical thinking

Mathematical thinking is about setting up formal systems of rules and examining their properties very closely. Proofs are a major component of mathematical thinking, which has a much more formal and unforgiving definition of proof than other fields. Computation has created a lot of new formal systems to study, and has been a fruitful area recently for mathematicians, just as physics was in previous centuries. Theoretical computer science is basically a branch of mathematics, involving primarily mathematical thinking.

scientific thinking

Scientific thinking studies the real world, constructing models of how it functions and testing the models empirically.  Different branches of science differ in how much they are model-driven and how much they are data-driven. Physics is highly model-driven, with the models often coming out 40 or 50 years in advance of the data (see Higgs boson).  Biology is highly data-driven often with non-quantitative  verbal stories as the models.  The key concept throughout science is empirical validation of predictive models.

engineering thinking

Engineering is about designing new things.  An engineering question is more of the form “how can I make this work?” rather than the science question “how does this work?”  I’ve talked about the distinction between science and engineering in one of my early blog posts, so I won’t belabor the point here.  Although scientists and engineers often wander back and forth between scientific and engineering thinking, the two are really distinctly different modes of thought.

systems thinking

Systems thinking is an important component of engineering thinking, consisting of dividing difficult problems into simpler subproblems with clearly defined interactions between the subproblems.  But systems thinking cuts across many fields, including mathematical thinking and scientific thinking. 

Computer programming is one of the best subjects to teach systems thinking in, because computer languages provide formal (though still inadequate) ways of representing the modules that encapsulate the subproblems and the interactions between them.  Electrical engineers try to do the same thing with their block diagrams, but these formalize a little less of the interactions, relying on English-language descriptions that are often omitted or poorly written. 

Unfortunately, many of the lower-level computer science classes have the faculty or textbook authors do all the systems thinking for the students, so that the students never learn to do it themselves. The scaffolding put in place to help the students find good solutions is where all the systems thinking happened, and descaffolding so that students have to learn to subdivide difficult problems into easier ones is an essential, but often missing, component of teaching programming.

The “multiple levels of abstraction” mentioned in the CMU definition of computational thinking is really about systems thinking, as each subproblem gets broken down into still smaller problems. 

algorithmic thinking

Algorithmic thinking is coming up with very precise recipes for doing things—not just flailing around trying things, but having very specific methods that can be shown to work (and work efficiently). Algorithmic thinking is really a branch of mathematical thinking, interested in provably correct manipulations in formal rule systems.  Originally it was applied to computing numerical functions, first manually and later by machine, but now has been expanded to cover many different types of data that can be represented in computers.  This is the second part of the CMU definition of computational thinking.

computational thinking

I don’t like the CMU definition of computational thinking, as they seem to have picked up definitions of mathematical, systems, and algorithmic thinking, and missed the computational part entirely. Computational thinking, to me, involves using computation to solve problems (data analysis, algorithmic solution of symbolic problems, numerical simulation, …) and may not involve much systems thinking or algorithmic thinking—someone else may have done that for you to enable you to use a computational tool.  Using Google to search for information is computational thinking, albeit at a rather low level.

statistical thinking

Statistical thinking is distinct from all of the above, though it is often important in scientific thinking.  Statistical thinking involves reasoning about data that comes from random processes, or that can be modeled as having been corrupted by random noise.  Notions of probability, sample size, statistical significance, multiple-hypothesis correction, correlation, and causation are all part of statistical thinking, which has applications to decision making in all aspects of life.

Obviously, there are overlaps and intersections between these different modes of thought (proofs done with the aid of a computer are a combination of mathematical and computational thinking, for example), but there are important differences also.  For example, Engineering thinking is not just systems thinking, but includes attention to fine details in the edge conditions (a hallmark of mathematical thinking), making allowances for variations in manufacturing (statistical thinking), testing how the device works in the real world (scientific thinking), and, very often these days, figuring out how to use cheap microcontrollers to do tasks that traditionally were done with more expensive analog devices (computational thinking).

The UCSC general education requirements (see my blog post on general education) recognize mathematical reasoning, scientific inquiry, and statistical reasoning as distinct parts of general education, adding textual analysis and cross-cultural analysis to cover what is often lumped under “critical thinking”.  They did not include anything that guarantees exposure to systems thinking, and they tossed in a few other things, some of which seem to me to be more politically expedient choices or fashion following than fundamental modes of thinking, but a general education system is always a compromise between different beliefs about what a university education should mean.  I think they did a better job of defining the basis for their general education system than most universities manage.

There have been a lot of calls for more education in “critical thinking” lately.  I’m not really happy with these calls, because teaching only a weakened version of the medieval trivium instead of more powerful modern forms of thinking does not educate students properly.

 

 

Grading based on a fixed “percent correct” scale is nonsense

On the hs2coll@yahoogroups.com mailing list for parents home-schooling high schoolers to prepare for college, parents occasionally discuss grading standards.  One parent commented that grading scales can vary a lot, with the example of an edX course in which 80% or higher was an A, while they were used to scales like those reported by Wikipedia, which gives

The most common grading scales for normal courses and honors/Advanced Placement courses are as follows:

	“Normal” courses		Honors/AP courses
Grade	Percentage	GPA	Percentage	GPA
A	90–100	3.67–4.00	93–100	4.5–5.0
B	80–89	2.67–3.33	85-92	3.5–4.49
C	70–79	1.67–2.33	77-84	2.5–3.49
D	60–69	1.0–1.33	70-76	2.0–2.49
E / F	0–59	0.0–0.99	0–69	0.0–1.99

​Because exams, quizzes, and homework assignments can vary in difficulty, there is no reason to suppose that 85% on one assessment has any meaningful relationship to 85% on another assessment.  At one extreme we have driving exams, which are often set up so that 85% right is barely passing—people are expected to get close to 100%.  At the other extreme, we have math competitions: the AMC 12 math exams have a median score around 63 out of 150, and the AMC 10 exams have 58 out of 150.  Getting 85% of the total points on the AMC 12 puts you in better than the top 1% of test takers.  (AMC statistics from http://amc-reg.maa.org/reports/generalreports.aspx ) The Putnam math prize exam is even tougher—the median score is often 0 or 1 out of 120, with top scores in the range 90 to 120. (Putnam statistics from  http://www.d.umn.edu/~jgallian/putnam.pdf) The point of the math competitions is to make meaningful distinctions among the top 1–5% of test takers in a relatively short time, so questions that the majority of test takers can answer are just time wasters.

I’ve never seen the point of having a fixed percentage correct ​used institution-wide for setting grades—the only point of such a standard is to tell teachers how hard to make their test questions.  Saying that 90% or 95% should represent an A merely says that tests questions must be easy enough that top students don’t have to work hard, and that distinctions among top students must be buried in the test-measurement noise.  Putting the pass level at 70% means that most of the test questions are being used to distinguish between different levels of failure, rather than different levels of success. My own quizzes and exams are intended to have a mean around 50% of possible points, with a wide spread to maximize the amount of information I get about student performance at all levels of performance, but I tend to err on the side of making the exams a little too tough (35% mean) rather than much too easy (85% mean), so I generally learn more about the top half of the class than the bottom half.

I’m ok with knowing more about the top half than the bottom half, but my exams also have a different problem: too often the distribution of results is bimodal, with a high correlation between the points earned on different questions. The questions are all measuring the same thing, which is good for measuring overall achievement, but which is not very useful for diagnosing what things individual students have learned or not learned.  This result is not very surprising, since I’m not interested in whether students know specific factoids, but in whether they can pull together the knowledge that they have to solve new problems.  Those who have developed that skill often can show it on many rather different problems, and those who haven’t struggle on any new problem.

Lior Pachter, in his blog post Time to end letter grades, points out that different faculty members have very different understandings of what letter grades mean, resulting in noticeably different distributions of grades for their classes. He looked at very large classes, where one would not expect enormous differences in the abilities of students from one class to another, so large differences in grading distributions are more likely due to differences in the meaning of the grades than in differences between the cohorts of students. He suggests that there be some sort of normalization applied, so that raw scores are translated in a professor- and course-specific way to a common scale that has a uniform meaning.  (That may be possible for large classes that are repeatedly taught, but is unlikely to work well in small courses, where year-to-year differences in student cohorts can be huge—I get large year-to-year variance in my intro grad class of about 20 students, with the top of the class some years being only at the performance level of  the median in other years.)  His approach at least recognizes that the raw scores themselves are meaningless out of context, unlike people who insist on “90% or better is an A”.

 People who design large exams professionally generally have training in psychometrics (or should, anyway).  Currently, the most popular approach to designing exams that need to be taken by many people is item-response theory (IRT), in which each question gets a number of parameters expressing how difficult the question is and (for the most common 3-parameter model) how good it is at distinguishing high-scoring from low-scoring people and how much to correct for guessing.  Fitting the 3-parameter model for each question on a test requires a lot of data (certainly more than could be gathered in any of my classes), but provides a lot of information about the usefulness of a question for different purposes.  Exams for go/no-go decisions, like driving exams, should have questions that are concentrated in difficulty near the decision threshold, and that distinguish well between those above and below the threshold.  Exams for ranking large numbers of people with no single threshold (like SAT exams for college admissions in many different colleges) should have questions whose difficulty is spread out over the range of thresholds.  IRT can be used for tuning a test (discarding questions that are too difficult, too easy, or that don’t distinguish well between high-performing and low-performing students), as well as for normalizing results to be on a uniform scale despite differences in question difficulty.  With enough data, IRT can be used to get uniform scale results from tests in which individuals don’t all get presented the same questions (as long as there is enough overlap in questions that the difficulty of the questions can be calibrated fairly), which permits adaptive testing that takes less testing time to get to the same level of precision.  Unfortunately, the model fitting for IRT is somewhat sensitive to outliers in the data, so very large sample sizes are needed for meaningful fitting, which means that IRT is not a particularly useful tool for classroom tests, though it is invaluable for large exams like the SAT and GRE.

The bottom line for me is that the conventional grading scales used in many schools (with 85% as a B, for example) are uninterpretable nonsense, that do nothing to convey useful information to teachers, students, parents, or any one else.  Without a solid understanding of the difficulty of a given assessment, the scores on it mean almost nothing.

Banana Slug genome crowd funding

T-shirt design from the first offering of the class. (click for high-res image)

A few years ago, I taught a Banana Slug Genomics course, based on some sequencing done for free as a training exercise for new technician.  I’ve mentioned the course occasionally on this blog:

• Cyberslug t-shirt designs
• Banana slug mitochondrial genome done (almost)
• More on the banana slug mitochondrion
• UCSC tiptoes around crowd funding for science

The initial, donated sequencing runs did not produce enough date or high enough quality data to assemble the genome to an annotatable state, though we did get a lot of snippets and a reasonable estimate of the genome size (about 2.3GB total and about 1.2GB unique, so a lot of repeats).  All the class notes are in a wiki at https://banana-slug.soe.ucsc.edu/) and the genome size estimates are at https://banana-slug.soe.ucsc.edu/bioinformatic_tools:jellyfish.

I did manage to assemble the mitochondrion after the class ended (notes at https://banana-slug.soe.ucsc.edu/computer_resources:assemblies:mitochondrion), but I now think I made a serious error in doing the assembly, treating variants due to a heterogeneous mitochondrial population as repeats instead.  The mitochondrion was relatively easy, because it is much shorter than the nuclear genome (probably in the range 23kB to 36kB, depending on whether the repeats are real) and has many more copies in the DNA library, so coverage was high enough to assemble it—the hard part was just selecting the relevant reads out of the sea of nuclear reads.

Ariolimax dolichophallus at UCSC, from larger image at http://commons.wikipedia.org/wiki/File:Banana_slug_at_UCSC.jpg

The banana slug genomics class has not been taught since Spring 2011, because there was no new data, and we’d milked the small amount of sequence data we had for all that we could get for it.  I’ve played with the idea of trying to get more sequence data, but Ariolimax dolichophallus is not the sort of organism that funding agencies love: it isn’t a pathogen, it isn’t a crop, it isn’t an agricultural pest, and it isn’t a popular model organism for studying basic biology. Although it has some cool biology (only capable of moving forward, genital opening on the side of its head, penis as long as its body, sex for up to 24 hours, sometimes will gnaw off penis to separate after sex, …), funding agencies just don’t see why anyone should care about the UCSC mascot.

Obviously, if anyone is ever going to determine the genome of this terrestrial mollusk, it will UCSC, and the sequencing will be done because it is a cool thing to do, not for monetary gain.  Of course, there is a lot of teaching value in having new data on an organism that is not closely related to any of the already sequenced organisms—the students will have to do almost everything from scratch, for real, as there is no back-of-the-book to look up answers in.

At one point I considered asking alumni for donations to fund more sequence data, but our dean at the time didn’t like the idea (or perhaps the course) and squelched the plan, not allowing us to send any requests to alumni. When the University started getting interested in crowd funding, I started tentative feelers with development about getting the project going, but the development people I talked with all left the University, so the project fizzled.  I had a full teaching load, so did not push for adding starting a crowd-funding campaign and teaching a course based on it to my workload.

This fall, seemingly out of nowhere (but perhaps prompted by the DNA Day celebrations last spring or by the upcoming 50-year anniversary of UCSC), I was asked what it would take to actually get a complete draft genome of the slug—someone else was interested in pushing it forward!  I talked with other faculty, and we decided that we could make some progress for about $5k–10k, and that for $20k in sequencing we could probably create a draft genome with most of the genes annotated.  This is a lot cheaper than 5 years ago, when we did the first banana slug sequencing.

Although the top tentacles of the banana slug are called eyestalks and are light sensing, they do not have vertebrate-style eyes as shown in this cartoon. Nor do they stick out quite that much.

And now there is a crowd funding campaign at http://proj.at/1rqVNj8 to raise $20k to do the project right!  They even put together this silly video to advertise the project:

Nader Pourmand will supervise students building the DNA library for sequencing during the winter, and Ed Green and I will teach the grad students in the spring how to assemble and annotate the genome.  Ed has much more experience at that than me, having worked with Neanderthal, Denisovan, polar bear, allligator, and other eukaryotic genomes, while I’ve only worked on tiny prokaryotic ones. (He’s also more famous and more photogenic, which is why he is in the advertising video.) We’re both taking on this class as overload this year (it will make my 6th course, in addition to my over-300-student advising load and administrative jobs), because we really like the project. Assuming that we get good data and can assemble the slug genome into big enough pieces to find genes, we’ll put up a genome browser for the slug.

I’m hoping that this time the class can do a better job of the Wiki, so that it is easier to find things on it and there is more background information.  I’d like to make the site be a comprehensive overview of banana-slug facts and research, as well as detailed lab notebook of the process we follow for constructing the genome.

Everyone, watch the video, visit the crowd funding site, read the info there (and as much of the Wiki as you can stomach), and tell your friends about the banana-slug-sequencing effort.  (Oh, and if you feel like donating, we’ll put the money to very good use.)

Update 30 Oct 2014: UCSC has put out a press release about the project.

Update 31 Oct 2014: It looks like they’ve made a better URL for the crowd-funding project: http://crowdfund.ucsc.edu/sluggenome

Summer project 2

In Summer Project, I introduced the project I’ve been working on all summer: a “kit” for making dimmable LED lamps, consisting of

• a dimmer board that reads a potentiometer and converts it (non-linearly) to a pulse-width-modulated 9V output signal and
• LED boards that hook up to the two wires of the PWM input signal, and that can be run in parallel,

and I showed the custom desk lamp I made for my son.  That was a fairly quick build, because it needed to be functional, but did not need to be very pretty—my son likes the exposed-wire look.  I asked him today how the lamp has been working out for the past week—he almost always uses it near the lowest setting, since he is either just filling in some shadows in an already lit room, or he is working at his desk while his roommate is sleeping.  In both cases he doesn’t need or want much light.  I suggested unmounting some of the LED boards, to get more control at the low light levels he needs.  Based on the measurements and calculations from the data sheets I did in LED board I-vs-V curve, he should be getting a range of  10–375 lumens from his desk lamp.  With only 3 LED boards, he would have a range of 6–225 lumens. But he likes having 5 shadows, so he turned down the suggestion. I considered changing the firmware for his lamp, to provide lower levels (has has all the equipment and software needed to reprogram it), but it is hard to get a duty cycle lower than 0.2% from the PWM controller.  If he really wants to go to low light levels, he could replace the 9V power supply with a 5V one, but then he’d have a range of 0.008–0.3 lumens, when what he probably wants is 1–40 lumens, which would need a 5.5V supply (not a standard size).  I think that he’ll sometimes need the 200–375 lumen range for task lighting when he is working with something fiddly late at night, so he is probably best off keeping with a full-power lamp.

The other project I mentioned was making a prototype table lamp for my sister, which needs to look a bit nicer, since she is considering making a series of table lamps using stiffened-silk shade.  I’ve now finished a prototype to send her, pictured below:

Here is the lamp, turned off. The base is a wooden bowl from the thrift store, sanded so that it sits flat. The upright is a standard brass lamp pipe, and the shade is just folded out of a 0.5m square of paper (the most common fold for making a paper cup).

When turned on, the lamp produces a modest downward light and illuminates the shade.

The lamp is done except for a knob for the potentiometer for controlling the dimmer. The only knobs I have are too large and industrial looking—I’ve ordered some smaller metal ones via Amazon, but they are being shipped from China, so I’ll probably have to mail my sister the lamp before the knobs get here. It turns out that if you want decorative, rather than ugly plastic, potentiometer knobs, the best source is companies that provide guitar parts.  The knobs for controls on electric guitars come in a wide variety of styles, some of them quite elegant.  (But guitar parts are also a fairly expensive way to get knobs, so make sure that you really like them!)

When I first assembled the lamp, there was a rather nasty flaw in the design, resulting in unintended shadows on the shade:

At first there was an extra shadow in the middle of the shade that I did not like.

With the shade off, it is easy to see where the extra shadow come from—it is the knurled nut connecting the up-facing LED board to the power wires.

The fix was easy—I just put the screw in from the top of the board, so that there was no large assembly to cast shadows:

Here is a closeup of the top part of the lamp, showing the top LED board facing up with the knurled nut on the back of the board. The two end LED boards face down, again having the knurled nut on the back, along with the heat sink.
I had originally planned to support the shade with the same 10-gauge copper wires that power the boards, but I realized that the cooper would corrode in a humid atmosphere, which might stain the shade, so I made a support out of 1/8″ 316L stainless steel welding rod, using a little hot-melt glue to attach pony beads to the ends, so that the rods wouldn’t poke holes in the shade.

The shadows cast by the LEDs with the corrected orientation of the screws is much cleaner than before.

I still have to write artist-level instructions on how to put together the electronics for a lamp. That will probably require a few more closeups of the lamp (with better lighting), which I’ll take during the week, before shipping the prototype to my sister.

Tread Lightly with Terra Nova!

A friend of mine, Ken Foster, has just started an Indiegogo fund-raising campaign, Tread Lightly with Terra Nova!, to raise money to restart the bicycle landscaping service that he ran for over 20 years:

Terra Nova’s Tread Lightly Service

Bicycle Powered Landscaping

In 1991 I started a service I dubbed Tread Lightly. This was a bicycle-powered landscape service that served our Santa Cruz area clients. For over twenty years the community hailed the ‘Tread Lightly’ service as an authentic, profoundly ecological approach to landscape care and as a symbol of innovation and hope. One of the principles of permaculture is “Use Small and Slow Solutions.” Tread Lightly was definitely that! Pedal-Powered Permaculture!

I used the Tread Lightly service for a couple of years, then decided that I did not care enough about the lawn to hire a landscape service—not even a bicycle landscape service from a friend. The service was good when I used it, and I was wondering why I never saw his bike trailers around town any more (I still see Ken riding his recumbent around town). It turned out that the trailers, bikes, and equipment eventually wore out, and the bike service was barely making enough to pay the employees. When the recession hit, it hit the local lawn services pretty hard, and Ken had to downsize his business (still doing landscaping, but with a much smaller team and no bikes).

Now that the local economy has improved, he’d like to bring back the signature bike trailers and hand equipment, but he needs to raise some capital to do it.  Borrowing from banks (a traditional business solution) is not likely to work, as the business plan does not result in a high probability of a large profit to pay off the loans. So he is looking for crowd-funding to help him restart the bike landscaping business, train youngsters in sustainable urban landscaping, and bring back a distinctive Santa Cruz institution.

In the years since I used the Tread Lightly service, I’ve bought an electric mower to mow the front yard about every 2 months, and let the back yard get covered with weeds (thistles, grass, blackberry brambles, ivy, kiwi vines, … ).  It is now difficult even to get to the compost heap, and some of the windows on the house are not openable because of the blackberries covering them.  I’ve been thinking of hiring Ken to clear the back yard for me, though I’ve no intention of actually maintaining the yard—too much work for too little reward.  If I were ever to sell the house, it would probably need over $1000 in landscaping maintenance to look attractive to a buyer, but I’m likely to be living here for the next 25 years, so any investment I make needs to pay off in personal pleasure (or reduced maintenance effort) well before then.

I enjoyed doing some gardening as a child, and thought I would enjoy it as an adult when I bought the house, but it turns out that I never have the time or energy to do any gardening. There is always something more interesting or more urgent to do. Even the tall raised beds that I built and that I had Ken build have gotten covered with weeds. It would be nice to have an herb and vegetable garden in them again, but I know I’ll never get around to planting and weeding, much less the incessant watering that is needed to have anything less hardy than thistles survive around here.  (I put in a drip irrigation system once, but such systems need annual maintenance, which I never got around to doing, so it disintegrated years ago.)