One week into new quarter

We’re one week into the new quarter (10% of the way through!) and the course is going ok. Most of the students have finished the first-week lab, which consists of installing a lot of software and soldering headers onto a Teensy LC board.

The software they had to install was

• Teensyduino
• Anaconda Python
• PteroDAQ
• gnuplot
• Waveforms

Of course, each piece of software has its own installation idiosyncracies, different on Windows, macos, and Linux.  Some people even bumped into some problems because of running old versions of macos or Python (which were luckily cleared by upgrading to slightly newer versions).

The soldering was a bigger problem, because many students plugged in their cheap irons and left them on for a long time without tinning the tips.  The result was a sufficient build-up of corrosion that that they could not then tin the tips—even using a copper ChoreBoy scrubber to clean the tips didn’t help in some cases. In the in-person labs, I often spent most of the first week labs cleaning soldering iron tips that students had managed to mess up, but I can’t do that online.  This was not such a problem last quarter, as most of the students knew how to care for soldering irons from the first half of the course, but it may be a bigger problem this quarter, as most of the students have never touched a soldering iron before.  Some of the ones who are living here in town may be contacting the lab staff to see if they can get access to tip tinner or get some help cleaning their irons.  Those further away may be buying tip tinner on their own—I had not included it in kits, because I nad not expected so many to need it and it costs $8 apiece.

Grading is going fairly well.  My grading team and I have had two Zoom meetings so far (for Homeworks 1 and 2) and I graded Quiz 1 by myself, so we are keeping up with the grading.  He have Homework 3 and Prelab 2a (there is no Prelab 1) both due Monday morning, and we’ll try getting them graded Monday night.  We’re having to do most of our grading in the evening, because one of the graders is living in China, 15 time zones away, and none of us in California is an early morning person.

In other news, I’ve finally finished clearing the blackberries and ivy from behind the garage (a project I started about 2 years ago).  I’ll probably find some more when I cut back the kiwi vine (an annual winter project, in addition to frequent minor pruning during the summer).  I think I either need to get some female kiwi vines and an arbor for them or uproot the male kiwi.  There is really not much point to having just a male kiwi intent on taking over a big chunk of the yard.

There are still a lot of blackberry roots out there that will sprout new vines.  I’ll try uprooting them where I have access (not where they are coming through the cracks in the concrete), but I’ll probably have to do a monthly sweep of the yard to remove blackberries for the rest of my life in this house.

Checked tandem-duplicate words in book

I got all the spelling checks done in the book today, and I noticed a “the the” in the book, so I looked for all occurrences of that pair of words in the LaTeX files and fixed them.  I then decided to write a tandem-word finder and look for all tandem duplicate words in the LaTeX files.  There were about ten others.  I was only checking a line at a time, though, so I decided to also convert the PDF file to text and check that.  That found another 5 or 6 tandem duplicate words (which had crossed line boundaries in the LaTeX files, but not in the output PDF file).

There were a lot of false positives in the PDF file, because “the Thévenin” somehow got treated as if it had “the the” with a word boundary after the second “the”.  There were also a lot of places in tables where numbers were duplicated, or description lists where the item head.

What I’ve not decided yet is whether it is worth rewriting the program to look for duplicate words that cross line boundaries—the program would be a bit more powerful, but I’d need to keep track of the place in the file better to be able to pinpoint where the error occurs, as I would not want to point to a full page as the location of the error.

Here is the code I wrote (edited 2020 Sept 10 to include page or line numbers):

#!/usr/bin/env python3

import re
import sys
import io

import pdftotext	# requires installing poppler and pdftotext

tandem_str = r"\b(\S+\b)\s+\b\1\b"
tandem_re = re.compile(tandem_str,re.IGNORECASE)

def lines_of_input(filenames):
    if not filenames:
        for line in sys.stdin:
            yield "--stdin",line
    else:
        for filename in filenames:
            if filename.endswith(".pdf"):
            	with open(filename, "rb") as file:
                    pdf = pdftotext.PDF(file)
                    for pagenum,page in enumerate(pdf):
                        for line in io.StringIO(page):
                            yield f'{filename} page {pagenum}',line
            else:
                with open(filename, 'r') as file:
                    for linenum,line in enumerate(file):
                        yield f'{filename} line {linenum}',line


for filename,line in lines_of_input(sys.argv[1:]):
#        print("DEBUG:", filename, line, file=sys.stderr)
        if tandem_re.search(line) is not None:
            print(filename,":",line.strip())

	

Checked URLs in book

I got all the URLs in my book checked yesterday.  Writing a program to extract the links and test them was not very difficult, though some of the links that work fine from Chrome or Preview mysteriously would not work from my link-checking program.

As it turns out, my son was writing me a link-checking program at the same time. His used pdfminer.six instead of PyPDF2, and relied on new features of Python (I still had Python 3.5.5 installed on this laptop, and f-format strings only came in with Python 3.6). I had to install a new version of Python with Anaconda to get his program to run. One difference in our programs is that he collected all the URLs and reduced them to a set of unique URLs (reducing 259 to 206), while I processed the URLs as they were encountered. His program is faster, but mine let me keep track of where in the book the URL occurred.

The checks we did are slightly different, so the programs picked up slightly different sets of bad URLs. He did just a “get” with a specified agent and stream set to True, while I tried “head”, then “get” if “head” failed, then “post” if “get” failed, but with default parameter values.  We also had different ways of detecting redirection (he used the url field of the response, while I used headers[“location”]), which got different redirection information. It might be worthwhile to write a better check program that does more detailed checking, but this pair of programs was enough to check the book, and I don’t want to waste more time on it.

I had to modify a number of the URLs for sites that had moved—in some cases having to Google some of the content in order to find where it had now been hidden. I wasted a lot of time trying to track one source of information back to a primary source, and finally gave up, relying on the (moved) secondary source that I had been citing before.

A surprising number of sites are only accessible with http and not https, and I ended up with eight URLs that I could not get to work in the link-check program, but that worked fine from the PDF file and from Chrome. Some of them worked from my son’s program also, but his failed on some that mine had success with.

Here is the code I wrote:

#!/usr/bin/env python3

import PyPDF2
import argparse
import sys

import requests	


def parse_args():
    """Parse the options and return what argparse does:
        a structure whose fields are the possible options
    """
    parser = argparse.ArgumentParser( description= __doc__, formatter_class = argparse.ArgumentDefaultsHelpFormatter )
    parser.add_argument("filenames", type=str, nargs="*",
            default=[],
            help="""names of files to check
            """)
    options=parser.parse_args()
    return options

    
def pdf_to_urls(pdf_file_name):
   """yields urls used as hyperlinks in file named by pdf_file_name
   """
   pdf = PyPDF2.PdfFileReader(pdf_file_name)
   for page_num in range(pdf.numPages):
        pdfPage = pdf.getPage(page_num)
        pageObject = pdfPage.getObject()
        if '/Annots' in pageObject.keys():
            ann = pageObject['/Annots']
            for a in ann:
               u = a.getObject()
               if '/URI' in u['/A']:
                   yield( page_num,  u['/A']['/URI'])


# HTTP status codes from https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
HTTP_codes = {
    100:"Continue"
    , 101:"Switching Protocol"
    , 102:"Processing (WebDAV)"
    , 102:"Early Hints"
    , 200:"OK"
    , 201:"Created"
    , 202:"Accepted"
    , 203:"Non-Authoritative Information"
    , 204:"No Content"
    , 205:"Reset Content"
    , 206:"Partial Content"
    , 207:"Multi-Status (WebDAV)"
    , 208:"Already Reported (WebDAV)"
    , 226:"IM Used (HTTP Delta encoding)"
    , 300:"Multiple Choice"
    , 301:"Moved Permanently"
    , 302:"Found"
    , 303:"See Other"
    , 304:"Not Modified"
    , 305:"Use Proxy (deprecated)"
    , 306:"unused"
    , 307:"Temporary Redirect"
    , 308:"Permanent Redirect"
    , 400:"Bad Request"
    , 401:"Unauthorized"
    , 402:"Payment Required"
    , 403:"Forbidden"
    , 404:"Not Found"
    , 405:"Method Not Allowed"
    , 406:"Not Acceptable"
    , 407:"Proxy Authentication Required"
    , 408:"Request Timeout"
    , 409:"Conflict"
    , 410:"Gone"
    , 411:"Length Required"
    , 412:"Precondition Failed"
    , 413:"Payload Too Large"
    , 414:"URI Too Long"
    , 415:"Unsupported Media Type"
    , 416:"Range Not Satisfiable"
    , 417:"Expectation Failed"
    , 418:"I'm a teapot"
    , 421:"Misdirected Request"
    , 422:"Unprocessable Entity (WebDAV)"
    , 423:"Locked (WebDAV)"
    , 424:"Failed Dependency (WebDAV)"
    , 425:"Too Early"
    , 426:"Upgrade Required"
    , 428:"Precondition Required"
    , 429:"Too Many Requests"
    , 431:"Request Header Fields Too Large"
    , 451:"Unavailable for Legal Reasons"
    , 500:"Internal Server Error"
    , 501:"Not Implemented"
    , 502:"Bad Gateway"
    , 503:"Service Unavailable"
    , 504:"Gateway Timeout"
    , 505:"HTTP Version Not Supported"
    , 506:"Variant Also Negotiates"
    , 507:"Insufficient Storage (WebDAV)"
    , 508:"Loop Detected (WebDAV)"
    , 510:"Not Extended"
    , 511:"Network Authentication Required"
    }



options=parse_args()
for pdf_name in options.filenames:
    print("checking",pdf_name,file=sys.stderr)
    for page_num,url in pdf_to_urls(pdf_name):
        print ("checking page",page_num, url, file=sys.stderr)
        req = None
        try:
            req = requests.head(url, verify=False)      # don't check SSL certificates
            if req.status_code in [403,405,406]: raise RuntimeError(HTTP_codes[req.status_code])
        except:
            print("--head failed, trying get",file=sys.stderr)
            try: 
                req = requests.get(url)
                if req.status_code in [403,405,406]: raise RuntimeError(HTTP_codes[req.status_code])
            except: 
                print("----get failed, trying post",file=sys.stderr)
                try: req = requests.post(url)
                except: pass
	
        if req is None:
            print("page",page_num, url, "requests failed with no return")
            print("!!!", url, "requests failed with no return", file=sys.stderr)
            continue

        if req.status_code not in (200,302):
            try: 
               code_meaning = HTTP_codes[req.status_code]
            except: 
               code_meaning = "Unknown code!!"
            
            try:
                new_url = req.headers["location"]
            except:
                new_url=url
            
            if url==new_url:
                print("page",page_num, url, req.status_code, code_meaning)
                print("!!!", url, req.status_code, code_meaning, file=sys.stderr)
            else:
                print("OK? page",page_num, url, "moved to", new_url, req.status_code, code_meaning)
                print("!!!", url, "moved to", new_url, req.status_code, code_meaning, file=sys.stderr)

More SolidWorks

I spent most of my day today with SolidWorks, fixing the problems noted in Bugs found in first assembly of robot and adding new layers to the robot. I’m getting a bit better at using SolidWorks, but I still find it to be an overly complicated interface with way too many modes.  I’m sure that there is a way to get it to start up with reasonable document parameters (like using mm instead of inches, or using the same settings as already open documents), but I’ve not taken the time to try to track that down.

Here is the model as it stands so far:

View from the front left of the robot. The octagon floating on top is a the beacon-detector board, which will be on standoffs that I didn’t bother to include in the model.

I cut out the three layers of the robot today, making two mistakes in the process. One mistake I caught right away, and just recut the layer after fixing the problem—there was an extra alignment circle that was not supposed to be cut that I had forgotten to erase. The other error was just as serious, but I didn’t notice it until I got home—the top layer did not have the slots cut in it for the spacers from layer 2 to layer 3. It is hard to notice this problem in looking at the SolidWorks model, as the 3D model looks the same whether the slots are cut or not. I should have noticed it when I created the dxf file for cutting the third layer, but by then I was getting pretty tired and careless. I’ll have to cut another copy on Thursday.  Luckily MDF is cheap—each layer costs me about $1, and I still have half a dozen 1-foot squares of MDF left.

Incidentally, I came up with what I think will be a cheap fix for the potential problem of the bumper springs not being stiff enough.  I added another switch front and center, just for the spring in the switch to push the bumper forward.  At 60¢ a switch, this is not a particularly expensive way to add a spring, and it saved me a lot of modeling and building time.  I could even wire up the switch if I can think of a use for it.

One other thing I made today was a “drill” test, to see what size holes were really made by the laser cutter from specifications.  I created the guide in SVG using a short Python program (so that I could tweak things easily. It took me quite a while to get the SVG just right, because of weird limitations of SVG, like that the path commands can’t take units for the coordinates. Also because I was using Inkscape to translate the SVG to the DXF format that the RDWorks laser-cutter software needs, and Inkscape assumes that the “pixels” are 90/inch for that conversion.  It is kind of messed up that SVG works in terms of “pixels”, since it is supposed to be Scalable Vector Graphics.  Inkscape only converts paths to DXF (not other shapes, like circles and text), so I wrote the program to generate paths and used Inkscape’s object-to-path conversion to convert the text.

Here is the piece I cut:

The circles were cut at 14mm/s and 100% (actually clipped at 67%) and the numbers were written at 140mm/s and 20%.

The holes were exactly the right size (to the 0.1mm limitations of my calipers), and the circular pieces that were cut out were 0.35–0.4mm smaller in diameter. That is, the kerf is about 0.19±0.02mm and it is on the inside of arcs.

Here is the code I used for generating the SVG file:

#!/usr/bin/env python

from __future__ import division, print_function

# all sizes are given in units of 0.1 mm

# Inkscape coverts pixels to real-world units in DXF at 90 pixels/in

pix_per_inch = 90
mm_per_inch = 25.4
pix_per_mm = pix_per_inch / mm_per_inch
pix_per_unit = 0.1*pix_per_mm

print('<?xml version="1.0" encoding="UTF-8" standalone="no"?>')
print('<svg width="150mm" height="150mm" xmlns="http://www.w3.org/2000/svg">');

y=100   # y-position of first row of circles

x_space = 60    # spacing between circles
stroke = 1       # stroke-width

xmax = None     # largest value for x

text_space =35  # space from circle to label

for diams in [range(5,50,5), range(50,80,5), range(80,105,5), range(105,130,5)]:
    x=100 # left edge of first circle
    for diam in diams:
        x += diam/2
        print ('<path stroke="red" fill="none" stroke-width="1" d="M {sx},{sy} \
a {r},{r} 0,0,0 {r},{r} \
a {r},{r} 0,0,0 {r},-{r} \
a {r},{r} 0,0,0 -{r},-{r} \
a {r},{r} 0,0,0 -{r},{r} z" />'.format(sx=(x-diam/2)*pix_per_unit, sy=y*pix_per_unit,
                r=diam/2*pix_per_unit))

        print('<text x="{}" y="{}" stroke="blue" fill="blue" text-anchor="middle" font-family="Verdana" font-size="10">'.format(
                x*pix_per_unit,(y+diam/2+text_space)*pix_per_unit))
        print(diam/10)
        print('</text>')
        x += diam/2 + x_space
        last_diam=diam
        if xmax is None or x>xmax:
                xmax = x
    y+=2*last_diam+text_space

print('<path stroke="red" fill="none" stroke-width="1" d="M 0,0 h {} v {} h {} z" />'.format(
        xmax*pix_per_unit,(y-last_diam)*pix_per_unit,
        -xmax*pix_per_unit))

print ('</svg>')

Single-arm design probably won’t work

In yesterday’s post Preliminary design review in mechatronics, I contemplated using a single arm to put balls in both targets, with the arm vertical for the higher Ren-ship target and past vertical for the AT-M6 target.  I tried doing some calculations with a hand calculator, and it looked like this might work.  I then drew out the geometry, labeling all the dimensions, and wrote a program to check the constraints carefully.   The program did not accept the values I’d computed by hand, so I did some search in the program and had it report what constraints failed.

Here is a sketch of the geometry:

The pivot is X from the edge and Y from the floor. The arm is A long with a cup that is W wide and D deep. The target is S from the edge and between L and H high.
The top and bottom of the cup should fit inside the target range.

The program selected the arm length based on the Ren-ship target, then checked that the arm would meet all constraints for both targets.

Unfortunately, an arm long enough to reach the Ren-ship target is too long to hit the AT-M6 target, no matter where I put the pivot. The problem is that AT-M6 target gets very small if the ball is approaching it with the arm almost horizontal, and the inner edge and outer edge of the ball cup can’t both clear the edges of the target.

I simplified the code to check just one target, and I do have a range of pivot positions that would make the arm work for a each target.  For the Ren-ship target I’d need to have the pivot at least 4.7″ off the floor in order for the arm to be short enough to fit in the initial box, and 6.5″ off the floor if I want the rest position of the arm to be horizontal (assuming the pivot is 1″ from the edge).

For the AT-M6 target, I’d need the pivot to be at least 2″ off the floor and and no more than 5″ off the floor.  The shortest arm would be about 7″ long with the pivot 1″ from the edge and 4″ up, with a separation of 3.8″.  The longest arm would be 10.58″ with the pivot 1″ from the edge and 2″ up, with a separation of 6″.  The best arm length (giving the most flexibility in position of the pivot and separation) seems to be around 7.5″.  I’d have to play around a bit with the computations to take into account other constraints (like interaction with other parts of the robot).  I’ll have to build some track-wired detectors, to see if I can get good sensitivity at large separations (which allows sloppier tape following, since I can be further from going out of bounds).  If sensitivity of the track-wire detector is low, then I can either try to make the robot follow the tape more precisely and risk going out of bounds, or I can deploy the track-wire detector outside the original 11″ box.

Now that the robot is not going to use a single arm for both targets, the don’t have to face the same direction.  The high arm can be on top facing front, and the low arm can be on a lower layer, facing sideways (but there needs to be clearance for the low arm to swing through the higher layers).

Here is the program trying to find a single-arm solution:

#!/usr/bin/env  python
""" arm-swing.py
    Fri Nov 10 07:56:41 PST 2017 Kevin Karplus

        Computations to see whether a single arm can be used for both
        the AT-M6 and Ren-ship targets for the 2017 Mechatronics course.
        
        The arm is
                A inches long with a
                D inch deep cup
                W inches wide (not extending arm)
                R inches above the floor at rest.
        The arm pivots 
                X inches from the edge of the robot, 
                Y inches from the floor.
        The targets are
                S inches from the edge of the robot
                L inches from the floor at the bottom of the hole
                H inches from the floor at the top of the hole
       
        Known values:
                L=6, H=10 for AT-M6
                L=13, H=15 for Ren ship
                
        Computed values:
                B to tip of cup:  sqrt(A^2+D^2)
                C to bottom edge of cup: sqrt((A-W)^2+D^2)
        
        Constraints:
                sqrt(A^2 - (R-Y)^2) + X < 11" to fit in initial cube
                R + (R-Y)D/A  < 11"     height restriction

                sqrt(A^2 - (R-Y)^2) + X approx 9" to load balls
                
                sqrt((S+X)^2 + (L-Y)^2) <= C (bottom edge clears) sqrt((S+X)^2 + (H-Y)^2) >= B  (top edge clears)
                
                S for AT-M6:  2"<S<6"  (probably around 4")
                S for Ren ship:  0<=S<=0.3"
                
                # mounting space for servo
                min_mount_X <=X <= 11"-min_mount_X
                min_mount_Y <=Y <= 11"-min_mount_Y
                
                If 1 1/2" Schedule 40 PVC pipe used for cup, then W=1.9"
"""

from __future__ import division, print_function
from math import sqrt
from numpy import arange

min_mount_X = 0.5
min_mount_Y = 1

def ConstraintsMet(A=7, D=1, W=1.9, R=6, X=2, Y=6, S=0, L=13, H=15,
        reason=False):
    """Check all the constraints, returning True is constraints met,
        False if one fails.
        
        if reason, then print the first constraint that fails
    """
    
    if X<min_mount_X or 11-X<min_mount_X:   
        if reason: print("too close to edge")
        return False    # too close to edge for mounting servo
    if Y<min_mount_Y: if reason: print("pivot too low") return False # too low to mount servo if Y>11-min_mount_Y:
        if reason: print("pivot too high")
        return False    # too high to mount servo
    
    if S<0: if reason: print("overlap target") return False # edge intersects target rest_above_pivot = R-Y rest_width = sqrt(A**2 - rest_above_pivot**2) if rest_width+X >= 11:      
        if reason: print("too wide")
        return False    # too wide for initial box
    
    rest_tip_height = R + rest_above_pivot*D/A  # resting height of tip
    if rest_tip_height >= 11:
        if reason: print("too tall")
        return False    # too tall for initial box
    
    # Note: the ball-loading constraints can be modified by 
    # loading in a different position than the initial rest position
    if rest_width+X < 8 or rest_width+X>=9.5:
         if reason: print("loading at {:.2f}".format(rest_width+X))
         return False    # not enough space for ball loading
    
    # distance from pivot to tip of cup
    tip_to_pivot= sqrt(A**2 + D**2)
    pivot_to_top = sqrt((S+X)**2 + (H-Y)**2)
    if tip_to_pivot > pivot_to_top:
#        print("DEBUG: hits top Y={:.2f}, A={:.2f}, tip_to_pivot={:.2f}, pivot_to_top={:.2f}".format(Y,A,tip_to_pivot,pivot_to_top))
        if reason: print("doesn't clear top, tip_to_pivot={:.2f} pivot_to_top={:.2f}".format(tip_to_pivot,pivot_to_top))
        return False    # doesn't clear top of hole
    
    # distance from pivot to inner lip of cup
    inner_to_pivot= sqrt((A-W)**2 + D**2)
    pivot_to_bottom = sqrt((S+X)**2 + (L-Y)**2)
    if inner_to_pivot < pivot_to_bottom:
#        print("DEBUG: hits bottom Y={:.2f}, A={:.2f}, inner_to_pivot={:.2f}, pivot_to_bottom={:.2f}".format(Y,A,inner_to_pivot,pivot_to_bottom))
        if reason: print("doesn't clear bottom inner_to_pivot={:.2f}, pivot_to_bottom={:.2f}".format(inner_to_pivot,pivot_to_bottom))
        return False    # doesn't clear bottom of hole

    return True
        
# tops and bottoms of targets
H_Ren = 15
L_Ren = 13

H_ATM6 = 10
L_ATM6 = 6

# some fixed values
W = 1.9
D = 1
X = 1

# some values to explore
for Y in arange(2., 9.05, 0.1):
    for R in arange(Y, 11.05, 0.1):
        # set arm length to just clear top of hole when touching Ren ship
        pivot_to_top = sqrt(X**2+(H_Ren-Y)**2)
        A_long = sqrt(pivot_to_top**2 - D**2)
        
        # set arm length to just clear bottom of hole when touching Ren ship
        pivot_to_bottom = sqrt(X**2 + (L_Ren-Y)**2)        
        A_short = sqrt(pivot_to_bottom**2 -D**2) + W
        
        for A in [A_short, A_long]:
            if not ConstraintsMet(A=A, D=D, W=W, R=R, X=X, Y=Y,
                    S=0, L=L_Ren, H=H_Ren):
                continue
            print ("Y={:.2f} met Ren constraints".format(Y))
            for S in arange(2.,6.05, 0.2):
                if ConstraintsMet(A=A, D=D, W=W, R=R, X=X, Y=Y,
                      S=S, L=L_ATM6, H=H_ATM6, reason=True):
                    print ("X={:.2f}, Y={:.2f}, A={:.2f}, R={:.2f}, SATM6={:.2f}".format(X,Y,A,R,S))