Strong enough to trot a mouse on

I mentioned today to my wife that I needed to remove the tea ball from my tea before it got strong enough to trot a mouse across (the Irish standard for how strong tea should be, but not to my taste for Pu-erh tea).

She wanted to know whether it was always the same mouse trotting across teacups, and we together decided that it should be different mice, but from a pure-bred lab strain.

That leaves the question of which strain.  The two most popular lab strains are C57BL/6 and BALB/c, so the question is which strain. The weights of C57BL/6J mice are given at https://www.jax.org/jax-mice-and-services/strain-data-sheet-pages/body-weight-chart-000664, and there is clearly a major difference by age and sex of the mouse. At five weeks, a female is about 17g, and a male about 20g, but at 24 weeks, the numbers are more like 25g and 35g. BALB/cJ mice seem to be slightly lighter: https://www.jax.org/jax-mice-and-services/strain-data-sheet-pages/body-weight-chart-000651.

So how old and which sex is the standard mouse for trotting across a cup of tea?  Or should the standard be based on just the weight of the mouse?  Does the size of the feet matter? Are there special strains of mice bred by Irish tea companies for testing their tea?

 

Rye bread rolls again

I managed to resurrect my sourdough starter by adding more yeast and keeping it in the refrigerator for a week, and I made a whole-wheat sourdough bread that came out pretty good (good crust and crumb, but not sour enough for my taste). It was basically the same ingredients and amounts as bread-machine bread, except for starting with sourdough starter for part of the liquid, bread flour, and yeast, and substituting molasses for the sugar.  I also baked it on baking parchment (no loaf pan) for the first half of the baking time, then directly on the terracotta tiles for the second half.

The whole-wheat sourdough looked good, as well as tasting pretty good.

So this week I decided to do the rye bread rolls again—the recipe that originated my sourdough.  Now that I have a starter, the recipe is a little different:

Feed starter:

1 cup sourdough starter
½ cup warm water (105°–115°F)
½ cup Strauss yogurt (the sourest one in our local market)
1 cup rye flour

Mix together in bowl, cover with wet dish towel, and let sit at room temperature for a day.  Remove and refrigerate one cup of the mixture for future sourdough starter, and let the rest sit for a day or two more at room temperature.

Dough:

the aged sourdough starter
1¼ cup warm water (105°–115°F)
1 teaspoon yeast
1 tablespoon salt
1½ cup bread flour
3 cups rye flour
1½ cup raisins

Stir down the starter, blend in water, yeast and salt. Let sit for 3–4 minutes so yeast can dissolve.

Stir in bread flour.  Add rye flour a cup at a time until dough forms a mass.  Stir with silicone scraper until dough has lost most of its stickiness.  Stir in raisins (this is different from last time, when I didn’t add the raisins until just before shaping). Turn from the bowl onto floured surface.

Knead on well-floured surface until dough soft and elastic (about 6 minutes), adding about ¼ cup rye flour to keep dough from sticking.  May need to scrape surface initially, as dough starts out very sticky. Put in greased bowl, cover, and let rise for 1–2 hours.

Punch down, divide dough into 2-oz pieces, and roll into balls.  Place on baking parchment on baking sheets, cover, and let rise until double in size (about one hour). Makes 24 rolls.

Glaze:

1 egg yolk
1 tablespoon milk

Remove cover, brush each roll with glaze, and cut X into top of each roll. Bake in preheated 400°F oven for 35–40 minutes.  Done when browned on the bottom and feel solid when pinched.

Cool on wire rack.

The rye rolls came out looking good. They had a nice flavor and texture, but not as much sourness as I would like. I think I also need to cut the crosses deeper.

Edition 1.1 released today!

I finally released the new version of the textbook today!  (https://leanpub.com/applied_analog_electronics). The book is only slightly longer than the previous edition:

659 pages
337 figures
14 tables
515 index entries
162 references

The chapter on design report guidelines is available free as a separate publication:
https://leanpub.com/design_report_guidelines

At the same time as I released the new edition, I eliminated my COVID-19 sale, so the minimum price is now $7.99. I will still provide coupons for free copies to instructors who are considering using the textbook for a course.

I may have to do another version before January, as I have not checked the labs for BME 51A yet to see what modifications are needed for doing the labs at home. For example, I haven’t decided whether it is worth buying more blood-pressure cuffs and extra tubing, to have enough to ship one to everyone. I’ll probably have to give up on the drill-press instruction. I’d rather not skip the micrometer instruction, but that would mean buying a lot more micrometers, as we generally share 5 for the whole class.

One nice thing about selling through Leanpub is that purchasers get all future editions published through Leanpub as part of the price—the company is trying to encourage authors to publish book drafts through them, rather than waiting until the book is completely polished. That means that students who got earlier versions of the book will get this release for free, and anyone who buys now will get the benefit of future releases.

 

Checked tandem-duplicate words in book

I got all the spelling checks done in the book today, and I noticed a “the the” in the book, so I looked for all occurrences of that pair of words in the LaTeX files and fixed them.  I then decided to write a tandem-word finder and look for all tandem duplicate words in the LaTeX files.  There were about ten others.  I was only checking a line at a time, though, so I decided to also convert the PDF file to text and check that.  That found another 5 or 6 tandem duplicate words (which had crossed line boundaries in the LaTeX files, but not in the output PDF file).

There were a lot of false positives in the PDF file, because “the Thévenin” somehow got treated as if it had “the the” with a word boundary after the second “the”.  There were also a lot of places in tables where numbers were duplicated, or description lists where the item head.

What I’ve not decided yet is whether it is worth rewriting the program to look for duplicate words that cross line boundaries—the program would be a bit more powerful, but I’d need to keep track of the place in the file better to be able to pinpoint where the error occurs, as I would not want to point to a full page as the location of the error.

Here is the code I wrote (edited 2020 Sept 10 to include page or line numbers):

#!/usr/bin/env python3

import re
import sys
import io

import pdftotext	# requires installing poppler and pdftotext

tandem_str = r"\b(\S+\b)\s+\b\1\b"
tandem_re = re.compile(tandem_str,re.IGNORECASE)

def lines_of_input(filenames):
    if not filenames:
        for line in sys.stdin:
            yield "--stdin",line
    else:
        for filename in filenames:
            if filename.endswith(".pdf"):
            	with open(filename, "rb") as file:
                    pdf = pdftotext.PDF(file)
                    for pagenum,page in enumerate(pdf):
                        for line in io.StringIO(page):
                            yield f'{filename} page {pagenum}',line
            else:
                with open(filename, 'r') as file:
                    for linenum,line in enumerate(file):
                        yield f'{filename} line {linenum}',line


for filename,line in lines_of_input(sys.argv[1:]):
#        print("DEBUG:", filename, line, file=sys.stderr)
        if tandem_re.search(line) is not None:
            print(filename,":",line.strip())

	

Checked URLs in book

I got all the URLs in my book checked yesterday.  Writing a program to extract the links and test them was not very difficult, though some of the links that work fine from Chrome or Preview mysteriously would not work from my link-checking program.

As it turns out, my son was writing me a link-checking program at the same time. His used pdfminer.six instead of PyPDF2, and relied on new features of Python (I still had Python 3.5.5 installed on this laptop, and f-format strings only came in with Python 3.6). I had to install a new version of Python with Anaconda to get his program to run. One difference in our programs is that he collected all the URLs and reduced them to a set of unique URLs (reducing 259 to 206), while I processed the URLs as they were encountered. His program is faster, but mine let me keep track of where in the book the URL occurred.

The checks we did are slightly different, so the programs picked up slightly different sets of bad URLs. He did just a “get” with a specified agent and stream set to True, while I tried “head”, then “get” if “head” failed, then “post” if “get” failed, but with default parameter values.  We also had different ways of detecting redirection (he used the url field of the response, while I used headers[“location”]), which got different redirection information. It might be worthwhile to write a better check program that does more detailed checking, but this pair of programs was enough to check the book, and I don’t want to waste more time on it.

I had to modify a number of the URLs for sites that had moved—in some cases having to Google some of the content in order to find where it had now been hidden. I wasted a lot of time trying to track one source of information back to a primary source, and finally gave up, relying on the (moved) secondary source that I had been citing before.

A surprising number of sites are only accessible with http and not https, and I ended up with eight URLs that I could not get to work in the link-check program, but that worked fine from the PDF file and from Chrome. Some of them worked from my son’s program also, but his failed on some that mine had success with.

Here is the code I wrote:

#!/usr/bin/env python3

import PyPDF2
import argparse
import sys

import requests	


def parse_args():
    """Parse the options and return what argparse does:
        a structure whose fields are the possible options
    """
    parser = argparse.ArgumentParser( description= __doc__, formatter_class = argparse.ArgumentDefaultsHelpFormatter )
    parser.add_argument("filenames", type=str, nargs="*",
            default=[],
            help="""names of files to check
            """)
    options=parser.parse_args()
    return options

    
def pdf_to_urls(pdf_file_name):
   """yields urls used as hyperlinks in file named by pdf_file_name
   """
   pdf = PyPDF2.PdfFileReader(pdf_file_name)
   for page_num in range(pdf.numPages):
        pdfPage = pdf.getPage(page_num)
        pageObject = pdfPage.getObject()
        if '/Annots' in pageObject.keys():
            ann = pageObject['/Annots']
            for a in ann:
               u = a.getObject()
               if '/URI' in u['/A']:
                   yield( page_num,  u['/A']['/URI'])


# HTTP status codes from https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
HTTP_codes = {
    100:"Continue"
    , 101:"Switching Protocol"
    , 102:"Processing (WebDAV)"
    , 102:"Early Hints"
    , 200:"OK"
    , 201:"Created"
    , 202:"Accepted"
    , 203:"Non-Authoritative Information"
    , 204:"No Content"
    , 205:"Reset Content"
    , 206:"Partial Content"
    , 207:"Multi-Status (WebDAV)"
    , 208:"Already Reported (WebDAV)"
    , 226:"IM Used (HTTP Delta encoding)"
    , 300:"Multiple Choice"
    , 301:"Moved Permanently"
    , 302:"Found"
    , 303:"See Other"
    , 304:"Not Modified"
    , 305:"Use Proxy (deprecated)"
    , 306:"unused"
    , 307:"Temporary Redirect"
    , 308:"Permanent Redirect"
    , 400:"Bad Request"
    , 401:"Unauthorized"
    , 402:"Payment Required"
    , 403:"Forbidden"
    , 404:"Not Found"
    , 405:"Method Not Allowed"
    , 406:"Not Acceptable"
    , 407:"Proxy Authentication Required"
    , 408:"Request Timeout"
    , 409:"Conflict"
    , 410:"Gone"
    , 411:"Length Required"
    , 412:"Precondition Failed"
    , 413:"Payload Too Large"
    , 414:"URI Too Long"
    , 415:"Unsupported Media Type"
    , 416:"Range Not Satisfiable"
    , 417:"Expectation Failed"
    , 418:"I'm a teapot"
    , 421:"Misdirected Request"
    , 422:"Unprocessable Entity (WebDAV)"
    , 423:"Locked (WebDAV)"
    , 424:"Failed Dependency (WebDAV)"
    , 425:"Too Early"
    , 426:"Upgrade Required"
    , 428:"Precondition Required"
    , 429:"Too Many Requests"
    , 431:"Request Header Fields Too Large"
    , 451:"Unavailable for Legal Reasons"
    , 500:"Internal Server Error"
    , 501:"Not Implemented"
    , 502:"Bad Gateway"
    , 503:"Service Unavailable"
    , 504:"Gateway Timeout"
    , 505:"HTTP Version Not Supported"
    , 506:"Variant Also Negotiates"
    , 507:"Insufficient Storage (WebDAV)"
    , 508:"Loop Detected (WebDAV)"
    , 510:"Not Extended"
    , 511:"Network Authentication Required"
    }



options=parse_args()
for pdf_name in options.filenames:
    print("checking",pdf_name,file=sys.stderr)
    for page_num,url in pdf_to_urls(pdf_name):
        print ("checking page",page_num, url, file=sys.stderr)
        req = None
        try:
            req = requests.head(url, verify=False)      # don't check SSL certificates
            if req.status_code in [403,405,406]: raise RuntimeError(HTTP_codes[req.status_code])
        except:
            print("--head failed, trying get",file=sys.stderr)
            try: 
                req = requests.get(url)
                if req.status_code in [403,405,406]: raise RuntimeError(HTTP_codes[req.status_code])
            except: 
                print("----get failed, trying post",file=sys.stderr)
                try: req = requests.post(url)
                except: pass
	
        if req is None:
            print("page",page_num, url, "requests failed with no return")
            print("!!!", url, "requests failed with no return", file=sys.stderr)
            continue

        if req.status_code not in (200,302):
            try: 
               code_meaning = HTTP_codes[req.status_code]
            except: 
               code_meaning = "Unknown code!!"
            
            try:
                new_url = req.headers["location"]
            except:
                new_url=url
            
            if url==new_url:
                print("page",page_num, url, req.status_code, code_meaning)
                print("!!!", url, req.status_code, code_meaning, file=sys.stderr)
            else:
                print("OK? page",page_num, url, "moved to", new_url, req.status_code, code_meaning)
                print("!!!", url, "moved to", new_url, req.status_code, code_meaning, file=sys.stderr)