TwiceTwo.com

Universal environment variables with pam_env

Fri, 31 Mar 2017 00:00:00 -0700

One of the minor annoyances of working in Unix-land is the variety of ways to set user environment variables: .zshenv, .bash_profile, .profile, .xinitrc, and so forth. Depending on where you set them, your variables may be visible in interactive shells, login shells, X applications, or some strange mixture. Even worse is that some, but not all, environment variables are set when interpreting your crontab, so that has yet another set of rules you have to remember.

I’ve been gradually working on setting up my home directory so that it contains everything (barring system packages) that I need to work, so that migrating to a new computer is just a matter of installing packages and then copying my $HOME. This only works if I have a consistent way to set environment variables, so that they are visible to every program that runs under my account, regardless of how I logon. And I think I’ve finally got it.

This process requires changing a couple of system-wide settings, but only in special circumstances:

If you care about having environment variables enabled when you login via TTY or SSH, make the following change to /etc/pam.d/login: find the line
```
 session       required   pam_env.so readenv=1 envfile=/etc/default/locale
```
and change it to read
```
 session       required   pam_env.so readenv=1 user_readenv=1 envfile=/etc/default/locale
```
(In the default config, pam_env is loaded only when you login via your display manager: MDM on Mint, GDM on Gnome, etc.)
If you want to set LD_LIBRARY_PATH then you have to make a change to the Xsession settings: add a file 90preserve_ld_library_path to /etc/X11/Xsession.d/ containing
```
 STARTUP="/usr/bin/env LD_LIBRARY_PATH=${LD_LIBRARY_PATH} ${STARTUP}"
```
This is only necessary because X normally starts ssh-agent which clears `LDLIBRARYPATH for security reasons. Another option is to disable loading of ssh-agent entirely.

If you want your environment variables to apply to your crontab (and you do), edit /etc/pam.d/cron and find the line

 session       required   pam_env.so envfile=/etc/default/locale

with

 session       required   pam_env.so user_readenv=1 envfile=/etc/default/locale

After making that change(s), create a file ~/.pam_environment and add your variables to it like this:

# HOME is not defined when this file is executed, and BOOST_ROOT needs 
# to be set somewhere, so...
HOME       DEFAULT=/home/@{PAM_USER}
BOOST_ROOT DEFAULT=${HOME}/bin/boost

# Setup path
PATH DEFAULT=${HOME}/bin:${HOME}/bin/bin:${PATH}
PATH DEFAULT=${HOME}/bin/rakudo:${PATH}
PATH DEFAULT=/opt/texlive/2016/bin/x86_64-linux:${PATH}
PATH DEFAULT=/opt/teyjus:${PATH}
PATH DEFAULT=/opt/bin:${PATH}
PATH DEFAULT=${HOME}/.cabal/bin:${PATH}
PATH DEFAULT=${HOME}/.local/bin:${PATH}
PATH DEFAULT=/opt/cabal/1.20/bin:/opt/ghc/7.8.4/bin:${PATH}
PATH DEFAULT=${HOME}/bin/tmsu-x86_64-0.6.1/bin:${PATH}
PATH DEFAULT=${HOME}/bin/boost:${PATH}

# Python paths
PYTHONPATH DEFAULT=${HOME}/.local/lib/python/site-packages:${PYTHONPATH}
PYTHONPATH DEFAULT=${HOME}/.local/lib/python2.7/site-packages:${PYTHONPATH}

# Manpaths
MANPATH DEFAULT=/opt/texlive/2014/texmf/doc/man:${MANPATH}
MANPATH DEFAULT=${HOME}/docs/man:${MANPATH}
MANPATH DEFAULT=${HOME}/bin/man:${MANPATH}

# Infopaths
INFOPATH DEFAULT=/opt/texlive/2016/texmf/doc/info:${INFOPATH}

# Library/link paths
LD_LIBRARY_PATH DEFAULT=/usr/local/lib:${BOOST_ROOT}/stage/lib:${HOME}/lib:${HOME}/lib/sfml:${LD_LIBRARY_PATH}
LIBRARY_PATH    DEFAULT=${HOME}/lib:${HOME}/lib/sfml:${BOOST_ROOT}/stage/lib:${LIBRARY_PATH}

# Include paths
CPLUS_INCLUDE_PATH DEFAULT=${HOME}/include:${BOOST_ROOT}:${CPLUS_INCLUDE_PATH}

# TeX paths
TEXINPUTS DEFAULT=.:${HOME}/.texmf/tex/:
BIBINPUTS DEFAULT=.:${HOME}/.texmf/tex/:

This is my actual .pam_environment. Setting environment variables that refer to other, existing variables (e.g., extending $PATH) uses a fancy syntax:

NAME DEFAULT=...

Within the value, you can use ${...} to expand environment variables, and use @{...} to expand PAM_ITEMs (as I did to get the user’s login name, because $USER isn’t set yet).

After that, it’s basically a matter of setting things the way you want. Note that if you aren’t using any expansions, you can write just

NAME=value...

With this setup, all of my environment variables are available to any process that runs under my account, including cron jobs! No more fighting with my editor to get it to use my $PATH when it runs my build tools, and no more hard-coded paths in crontab, etc.

Backpacking the Kaiser Loop trail, postmortem

Mon, 20 Jun 2016 00:00:00 -0700

My girlfriend and I backpacked almost the entire Kaiser Loop trail over three days, June 15-17 (we took the shortcut through Mary’s Meadow, shaves 2.1 mi. off the entire loop). We traveled the loop clockwise, which allowed us to split the elevation gain (nearly 3000 ft.) over two days. We camped at Nellie lake the first night (8900 ft) and at Line Creek lake (9800 ft) the second, climbing to the summit of Kaiser Peak on our third day before heading down the mountain. Since I’ve already covered a bunch of backpacking stuff in other posts, I’ll just briefly describe some things that I learned.

Map and Route

Hiking with a partner

Hiking with a partner is a fantastic way to stave off insanity, provided, of course, that they are willing and generally enthusiastic about it. I mentioned how, when I backpacked alone, my mind went to some strange places. This time I had no “mental moments”; the closest was a few dreams about losing the trail (see below for the impetus for this).

Gear

I switched up a number of my gear choices this year, hoping to save weight.

I used a Gossamer Gear Mariposa backpack instead of my Deuter. The Mariposa has a smaller internal storage, which is only top-loading (no sleeping bag section with its own bottom zipper) which does require some discipline in packing. From bottom to top, I generally packed cookset, sleeping pad, sleeping bag (just stuffed into the pack), bear canister. The beauty of the Mariposa is the almost 20L of external storage it offers: I was able to pack my tent, water, water filtration, toilet kit, day clothes, and other items all into the external pockets for easy access and unpacking.
I switched my camera from an Olympus E420 to a Panasonic DMC-TS6R. This is an “outdoor” camera: waterproof, shockproof, with GPS, altimeter, and compass so that your photos are (sometimes) geotagged with their location. It always weighs 8 oz to the Olympus’s 20 oz.
I switched my shoes from New Balance Minimus Zero, to the Montrail Massochist III. I actually tried three different pairs of shoes before the trip: the La Sportiva Ultra Raptors were unwearably painful. Even going up a size (as is recommended) they were so tight as to make my feet numb after less than an hour of normal walking, and they had weird plastic protrusions on the inside that dug into my skin. The other shoes I tried were Skarpa Spark trail runners; they were fine, just not as comfortable as the Montrails.

The Montrail Masochist was great! It has a mesh upper so it (and my socks inside) dried quickly after getting wet. The tread is good and grippy, and performed well on wet rocks and snow. They do weigh more than the New Balances, but no where near as much as a full boot; the weight is about the same as the dress shoes I wear to work, so I didn’t really notice them. They did feel a bit tight around the heel/Achiles tendon (as people in Amazon reviews have complained about) but this was temporary; an hour of walking broke them in and flexed up that part of the shoe so that I had no discomfort at all. (I ended the trip with zero blisters or hot spots on my feet.)
I was forced to switch my water filtration from the Sawyer Squeeze to Aqua Mira, after my Squeeze decided to not work at all the first time I tried to use it (note to self: always test water filter before leaving!). Aqua Mira takes longer than the Squeeze (5 mins for the chemicals to “activate” and then 30 mins for it to purify) and thus requires some scheduling and coordination. Generally I had two water bottles: one was in the process of being purified, while the other was drinkable. When necessary, I’d fill my girlfriend’s water bottle/pouch from my drinkable bottle. Whenever we hit a stream, we’d redistribute drinkable water and then refill any empty bottles with water to be purified as we walked. This worked well, and I’m glad I had the Aqua Mira as a backup.

Note that although the instructions don’t say so, you can in fact use 7-8 drops of “A” and “B” for each liter to filter more than 1 liter at a time. They also don’t mention that you should try to keep the time the mixture spends activating as close to 5 mins as possible; less or more means that the active ingredient will not be as concentrated and as a result, purification may take longer/not be as effective.

Snow

The middle of June is still “early” in the summer hiking season, at least for the higher elevations. Above 9000 ft we had some areas with substantial snow to deal with. Most of our problems came when the trail was covered for more than a few yards; there were several places where we had no idea where the trail was going and had to just forge ahead, hoping to find it once we got past the snow.

The most difficult section was just before and after the unnamed lake that lies at the foot of the switchbacks leading up to the ridge. Below the lake, the trail follows the northern slopes (something that was not shown on my map) which was more snow-covered, shaded, and steeper, than the crest. At one point the trail was covered by a patch of snow that would have been dangerous to cross; a slip would have sent us sliding down the side of the mountain. We went off-trail and tracked around it until we picked up the trail the rest of the way to the lake.

Beyond the lake, the trail switchbacks up to the ridge; about halfway up the trail was entirely covered in snow. We essentially just headed straight up, climbing over rocks when possible but often crossing the snow, until we reached the top and picked up the trail again.

There were a few patches of snow on the way down, but these were smaller and well-traveled, so that there was no doubt as to where the trail went.

Food

We did great for food! Our dinners were supplied by Packit Gourmet (gumbo and chili). Their meals ostensibly “serve one” but are big enough that a single meal can be extended to serve two without much trouble. E.g., to pad out the gumbo we just added half a cup of instant rice and some summer sausage. I carried a BearVault 500, but during the day my girlfriend kept a separate “snack bag” in the top of her backpack for easy access. (At night we’d pack the whole snack bag into the bear canister, obviously.) Each morning we’d put whatever we intended to have for lunch into the snack bag before loading up.

Our breakfast consisted of precooked bacon (reheated), coffee, and oatmeal with freeze-dried fruit. (Note that freeze-dried strawberries rehydrate much faster than either blueberries or raspberries.)

Lunch was sandwhiches from Subway (day 1), dehydrated pasta salad with salmon (day 2), and cheese and crackers with summer sausage (day 3). Note that many people will advise you to wrap your cheese in wax paper so it can “breathe”, suggesting that it will develop a dry skin. They’re right, but why not let it develop that skin before you leave? Wrap it in wax paper and put it in the fridge a couple days before you leave.

Dinner was gumbo with extra rice and sausage (day 1) and chili with cracker bread and bacon (day 2). Speaking of which, Armenian cracker bread is a fun alternative to tortillas. Despite the name, it’s not crispy but is actually quite flexible. You can use it as a kind of edible towel to clean out your pot after a meal.

Windows Protocol Handlers for Sysadmins

Wed, 08 Jun 2016 00:00:00 -0700

On Windows, it’s possible to create new “protocol handlers” so that URLs like ssh://hostname, telnet://hostname, etc. are recognized and launch some application (e.g., PuTTY). The only trickery is that Windows will pass the entire protocol string to the application (e.g., PuTTY will be launched with the command line putty telnet://hostname). For applications that understand the protocol prefix this is fine, but otherwise we have to employ a bit of scripting trickery to strip it off. Following this SO answer led to the possibility of using protocol handlers for all kinds of sysadmin things (the advantage being that protocol links can be embedded in wikis, Visio diagrams, hotsheets, inventory systems, etc.). As creating the required registry keys is a bit of a pain, I wrote regproto.js, a script for registering new protocol handlers.

Some examples should make its usage clear (run these from a command prompt, in the folder where you saved regproto.js):

regproto.js ping     "ping -t %%HOST%%"
regproto.js telnet   "putty -telnet %%HOST%%"
regproto.js ssh      "putty -ssh %%HOST%%" 
regproto.js cmd      "psexec \\%%HOST%% cmd.exe"
regproto.js shutdown "shutdown /s /m \\%%HOST%%"
regproto.js restart  "shutdown /r /m \\%%HOST%%"
regproto.js manage   "mmc compmgmt.msc /computer=\\%%HOST%%"
regproto.js ps       "powershell -NoExit -Command Enter-PSSession -ComputerName %%HOST%%"
regproto.js rdp      "mstsc /v:%%HOST%%"

The above will register

A ping:// handler which will open a console window with ping -t running on the host.
The usual ssh:// and telnet:// handlers, linked to PuTTY (assuming it’s in your %PATH%).
A cmd:// handler that will use psexec to launch a command interpreter on the remote machine.
A shutdown:// handler that will shutdown the remote machine.
A restart:// handler that will restart the remote machine.
A manage:// handler that will launch the Computer Management snapin connected to the remote machine.
A ps:// handler that will run Powershell on the remote machine.
A rdp:// handler that will launch a remote desktop connection to the target machine.

The only required arguments are the protocol name, and the command string to launch. The command string can refer to %%HOST%%, the “hostname” (or whatever) given in the protocol://hostname. Note that this could include a username and password, if your application understands those (PuTTY does, so you can do telnet://username@hostname and it will work).

The full documentation for regproto is in a big comment at the top of the file (pay special attention to the note at the bottom about custom protocols in Firefox and Chrome).

regproto will create a script urllaunch.js in %SystemRoot%\System32. This script is used to launch applications with the URL given, but with the protocol stripped off. It’s possible that (e.g.) antivirus might block the creation of this script, in which case you won’t be able to register any protocols.

Download regproto.js

Backpacking: Shoes

Wed, 08 Jun 2016 00:00:00 -0700

(Updated June 8, 2016)

Except for the off-trail mountaineers among us, the old 10 lb. hiking boots have fallen out of favor. With research¹ finding that weight² on your feet requires approx 5x the energy³ to move as weight carried elsewhere, there’s good reason to lighten our foot load.

My “go to” hiking shoes, when it’s not cold or wet, have been New Balance Minimus trail running shoes. A pair weighs less than 6 oz, and with all the mesh they breathe incredible well. I actually own two pair: one in garishly neon blue, and another in glow-in-the-dark white (style is the last refuge of the incompetent).

Although I normally wear a size 11.5, and that works fine for day hiking with a lighter pack, I think it’s too small for backpacking. Being on your feet all day, and carrying more weight, makes your feet both swell and “pancake” (flatten out). My right big toe was numb for weeks afterwards, from being pressed inside my shoe. Although most people recommend you get a “good” fitting shoe (because having your feet slide around in your shoes is a good way to get blisters) I think I’ll risk a 12, 12.5 or 12W next time.

I had no problems with stability, “rolling” an ankle, etc. Of course, I wasn’t carrying an overwhelming amount of weight (30lbs), and I’ve never had any trouble with my ankles anyway, so your mileage may vary. Still, I personally am firmly in the camp of those who say that big boots with thick soles that lace up practically to your knees are actually worse for your ankles: the make your ankles weaker by not allowing you to exercise them, and the thicker soles and tread make it so that if you do roll an ankle, the displacement will be much greater than with a more minimal shoe, making it that much more likely that you’ll sustain a real injury.

That said, the Minimuses (Minimusi?) had two main disadvantages when it comes to backpacking:

The tread does not have much grip at all. I would be hesitant to wear them on any kind of slick or wet rock, and they were pretty unstable the few times I had to wade across streams in them.
The pair I wore got pretty torn up over the three days I was on them. They have since been relegated to lawn-mowing shoes. The tread is worn down and torn in several places, and, being white, the dirty stains are pretty much permanent. I would not trust them to hold up over a longer trip.

One nice thing about the Minimus is that they can be worn fairly comfortably without socks (I wore them with a pair of liner socks), and they are easy to slip on. This makes it practical to put them on in the middle of the night (glow-in-the-dark!) to answer the call of the wild without undue effort.

For a later trip, I tried a pair of La Sportiva Ultra Raptor trail runners, but found them unwearably painful. Although they are known to run a size small, and I purchased them a size larger than normal, they were still incredibly tight, especially in the heel. After wearing them around the house for about an hour, my feet were numb. They also have plastic projections on the roof of the shoe that dug into my skin.

Energy cost of backpacking in heavy boots S. J. Legg , A. Mahanty, Ergonomics, vol. 29, iss. 3, 1986 ↩
The energy cost and heart-rate response of trained and untrained subjects walking and running in shoes and boots Bruce H. Jones, Michael M. Toner, William L. Daniels, Joseph J. Knapik, Ergonomics, vol. 27, iss. 8, 1984 ↩
Physiological strain due to load carrying in heavy footwear M. Holewijn, R. Hens, L. J. A. Wammes ↩

Per-user customization of GTK2/3 themes

Wed, 04 May 2016 00:00:00 -0700

GTK2 and 3 both use theme to control the appearance of applications that use them. While an application can override the theme settings in various ways, there are also per-user (and even per-application) ways to customize their appearance and behavior.

GTK2

For GTK2, theme config files normally reside in $XDG_DATA_DIRS/themes/theme-name/gtk-2.0/, so something like /usr/share/themes/theme-name/gtk-2.0/. The main configuration file is gtkrc, with subsidiary files in styles/. It’s worth taking a browse through the gtkrc of your favorite theme, to get a feel for how things work.

Per-user, the gtkrc can be overridden/augmented with the settings in ~/.gtkrc-2.0. This file is loaded after the gtkrc for your current theme, so it only needs to specify settings that are different. E.g., mine looks like

# Add steppers to scrollbars
style "scrollbar-style" {
    GtkScrollbar::has-backward-stepper = 1  
    GtkScrollbar::has-forward-stepper = 1
}
class "GtkScrollbar" style "scrollbar-style"

# Make sure scrollbars don't warp on click
gtk-primary-button-warps-slider = 0

Note that the method described below, for GTK3, will also work for GTK2: you can create a custom theme in ~/.themes/, import the main theme, and then it will be used in place of it.

Also note that in GTK2 the rc file covers both appearance and behavior. E.g., you can use it do to things like create or override keyboard shortcuts, etc.

GTK3

GTK3 themes live in $XDG_DATA_DIRS/themes/theme-name/gtk-3.0/, but now there are several files to look at: settings.ini contains various behavior settings. Then there are probably a bunch of *.css files; GTK3 loads gtk.css, which usually imports a bunch of other files that touch on specific aspects of the system.

If you want to override settings.ini, just put your own in /.config/gtk-3.0/; this file will be loaded after the theme’s settings, so anything you do here will replace the theme’s settings. Mine looks like

[Settings]
gtk-primary-button-warps-slider = false

Changing settings in the .css files is a bit more tricky. You can create ~/.themes/theme-name/gtk-3.0/gtk.css but if this file exists the main theme file will not be loaded at all. So you can’t just put your changes in it, or every GTK3 application will load with a mostly empty theme. Instead, import the main gtk.css theme you prefer to use at the top of your file, and then add any customizations below it. E.g., here’s what mine looks like:

/* Load the original theme */
@import url("/usr/share/themes/Mint-X/gtk-3.0/gtk.css");

/* Override scrollbar setting */
.scrollbar {
    -GtkScrollbar-has-backward-stepper: 1 !important;
    -GtkScrollbar-has-forward-stepper: 1 !important;
}

Supposedly you can also place a settings.ini in the same directory as your customized theme and it will be picked up.

Per-application customization

Both GTK2 and 3 will try to load some aspects of their theme from files specified in environment variables, if they are set. You can use this to override parts of the theme, or switch to a different theme entirely, on a per-application basis.

For GTK2, the variable GTK2_RC_FILES, if set, will cause all files in the colon-separated list to be loaded as if they were gtkrc files, in order. I believe that this overrides your per-user file as well, so if you want any per-user settings to be applied, you’ll have to add $HOME/.gtkrc-2.0 to the list, probably at the end.

For GTK3, GTK_THEME specifies the gtk.css file to be loaded. You can use this to either switch to a different theme entirely, or create your own gtk.css as above that imports a theme and then overrides a few settings. I’m not sure if there is a way to override settings.ini on a per-app basis.

For GTK3, it’s possible to discard themes entirely and go with the default (ugly) theme, by unsetting GTK_DATA_PREFIX. E.g.,

GTK_DATA_PREFIX= my-application &

References

Resource files: GTK+2 Reference Manual – Describes where resource (rc) files are found, their format, etc. Unfortunately, there does not appear to be a quick and easy reference to all possible per-widget settings.
GTK+ CSS: GTK+ 3 Reference Manual – Reference to the CSS used in GTK3.
Bindings: GTK3 – Reference to the CSS used for assigning key bindings to widgets.
GtkCssProvider: GTK3 – Describes what CSS files GTK3 will try to load at startup, and in what order.
Settings: GTK3 – Describes the paths GTK3 will search for settings.ini files. Note that the settings themselves are generally described in the documentation for the widgets to which they apply.

Buiding Grafx2 on Linux (Ubuntu) from source

Fri, 18 Mar 2016 00:00:00 -0700

There aren’t any instructions for building Grafx2 from sources on Linux (Ubuntu-derivatives, in particular), and although the process is not complicated or involved, I thought I’d document it, for posterity. Note that Grafx2 still uses SDL 1.x, not SDL2. If you try to build it after installing the SDL2 dev packages, you’re going to have a bad time.

sudo apt-get install libsdl1.2-dev \
                     liblua5.1-0-dev \
                     libpng-dev \
                     libfontconfig1-dev \
                     libsdl-image1.2-dev \
                     libsdl-ttf2.0-dev

svn co svn://pulkomandy.tk/GrafX2

cd Grafx2/trunk/src
make

If you’re like me and you don’t like installing binaries into /usr/local, then you can install into ~/bin by creating a directory (say, Grafx2) and then copying ../bin and ../share into it. The Grafx2 binary is hard-coded to look for its auxilliary files in ../share.

Processing: Locking the mouse

Tue, 01 Mar 2016 00:00:00 -0800

As part of an example for my game development students (all of my examples are in Processing), I needed to be able to get not just the mouse position, but mouse movement deltas, whenever the mouse was moved (I wanted to build an FPS-style mouselook system for them to play with). In some systems, you can access the raw offsets that are coming from the mouse, but Java (AWT and JOGL) are not one of those systems. Instead, the accepted method is to constrain the mouse cursor to the window, hide the mouse cursor, and then, every time the mouse moves, reset its position to the the center of the window. So long as the user never moves the mouse fast enough to hit the edges of the window, this will have the same effect as getting the raw data.

For a full-screen sketch, this is relatively easy, regardless of the renderer in use. You can use a java.awt.Robot to set the mouse position to (displayWidth/2,displayHeight/2) in the mouseMoved handler.

For a windowed sketch, things are more difficult. A robot will set the mouse position in screen coordinates, not window coordinates, so in order to center the mouse cursor in the window, you have to know the screen position of it (along with the size of any chrome around the edges of the window). This turns out to be surprisingly difficult to get, particularly if you’re using the JAVA2D renderer.

(I should mention that this method has only been tested with Processing 3.0.1 248. Due to the propensity of Processing’s developers for breaking backwards compatibility, it’s entirely possible that everything I’m saying here will be obsolete next week.)

Processing sketches used to have a global frame variable containing a reference to the AWT frame containing the sketch. As of P3.0, the variable still exists, but it’s not a real frame. A few of its methods have been overloaded to give error messages (intended to let the user know that its use is deprecated), but getLocation() always returns (0,0), while getLocationOnScreen() throws an exception (because the frame is not part of a displayed window hierarchy). In my mind, it would have been better to leave frame set to null, so that any use of it would trigger an error.

The trick then is to figure out how to talk to the “real” window containing the sketch. The method varies depending on whether you are using P2D or P3D:

For JAVA2D, the value of the global surface variable will be a PSurfaceAWT, which exposes a getFrame() method returning the normal AWT frame containing the sketch. Thus, you can do:

Point p = surface.getFrame().getLocationOnScreen();
robot.setMousePosition(p.x + width/2, p.y + height/2);

(There’s also a new (?) 2D renderer based on JavaFX; I haven’t looked at what kind of window it sets up.)

For P3D (and possibly P2D?) we’re in better luck, as the window is actually handled by JOGL, which has some support routines that do what we want directly. surface.getNative() will return a GLWindow which exposes public methods:

confinePointer(boolean) – to restrict the pointer to the window
warpPointer(int,int) – to re-center the pointer position (note that this takes window coordinates!)
setPointerVisible(boolean) – to hide the mouse cursor

So we can just do

import com.jogamp.newt.opengl.GLWindow;
...
GLWindow r = (GLWindow)surface.getNative();
r.confinePointer(true);
r.setPointerVisible(false);
...
r.warpPointer(width/2,height/2);

Processing has a “libraries” browser that lets you search for libraries other people have written, but what it really needs is an examples browser, with some kind of automated testing to make sure that examples continue to work in newer versions.

The life, death, and rebirth of my laptop

Fri, 26 Feb 2016 00:00:00 -0800

The Dell Precision M4500, which had already given me so much grief died this morning. It was working fine last night, but this morning it would simply not power on; no lights, beeps, no indication of life at all. (I’m thinking motherboard; even though I got myself back up and running, I’ll probably order a replacement, just to get it working again so I can add it to my ever-growing pile of working-but-unused laptops.) Since this was my primary school and development laptop, it was imperative that I get something working.

(Note that I wasn’t worried about data loss. I use Tarsnap and have been quite happy with it, and I was pretty sure the HD was OK in any case; I just needed a working computer, preferably one with all my stuff on it.)

I still had my old Thinkpad T61 at home (which the M4500 was intended to replace), I figured I’d try a simple HD swap. This usually works well on Linux installs with stock kernels, as the distros tend to include drivers for everything so that changing hardware isn’t an issue. In this case, however, I ran into two problems.

Things that did not work

Setting the boot flag on the protective MBR partition did not work. Setting the legacy boot flag on the Linux system partition did not work. Installing Grub did not work (on GPT Grub needs its own partition to work with, and I didn’t have room to create one).

GPT vs. MBR

The M4500 was built to boot via GPT and UEFI; the Thinkpad supports neither. The first step was to get it booting via the old MBR+BIOS. Although there are apparently ways to partition your HD so that both GPT and MBR booting are supported, I hadn’t done that when I set this system up. That left converting the GPT structures to MBR. I only had three partitions, so I didn’t expect any problems, but it’s still scary whenever you do something that involves blowing away the partition table and building it from scratch. I used a Linux live USB drive (running Mint 17.1, the same version I’m using on the desktop) and ran sgdisk. The r menu contains a g option to convert GPT to MBR. It does almost all the work for you: you only have to set the boot flag on your Linux system partition when it’s done.

Grub for UEFI vs. MBR

The newly-created MBR does not have any bootloader code in it, so I had to reinstall Grub. This isn’t hard to do from a Live install, but there were a few quirks. Most tutorials will tell you to mount your existing Linux, mount /dev, /proc and a few other things into it, and then chroot in to run grub-install. But in my case, I didn’t have the x86-pc Grub modules, just the UEFI modules. So I also had to mount in /usr/lib/grub from the flash drive. After that the Grub install/update worked fine. (After booting my new system, I installed the grub-pc package so that the modules would always be available. This also removed the UEFI modules and reinstalled Grub (again!) just to make sure everything is in sync.)

The full process

Here’s a transcript of the full process, run from a live USB drive, after converting from GPT to MBR (which is just a matter of running sudo gdisk, then r, g, then exit and set the boot flag with the utility of your choice).

mint@mint ~ $ sudo mkdir /mnt/hd        
mint@mint ~ $ sudo mount /dev/sda2 /mnt/hd
mint@mint ~ $ sudo mount --bind /dev /mnt/hd/dev
mint@mint ~ $ sudo mount --bind /dev/pts /mnt/hd/dev/pts
mint@mint ~ $ sudo mount --bind /proc /mnt/hd/proc      
mint@mint ~ $ sudo mount --bind /sys /mnt/hd/sys     
mint@mint ~ $ sudo mount --bind /usr/lib/grub/ /mnt/hd/usr/lib/grub/
mint@mint ~ $ sudo chroot /mnt/hd
mint / # grub-install /dev/sda
Installing for i386-pc platform.
Installation finished. No error reported.
mint / # grub-install --recheck /dev/sda
Installing for i386-pc platform.
Installation finished. No error reported.
mint / # update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.13.0-37-generic
Found initrd image: /boot/initrd.img-3.13.0-37-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
  No volume groups found
done
mint / # exit
mint@mint ~ $ sudo umount /mnt/hd/sys
mint@mint ~ $ sudo umount /mnt/hd/proc
mint@mint ~ $ sudo umount /mnt/hd/dev/pts
mint@mint ~ $ sudo umount /mnt/hd/dev    
mint@mint ~ $ sudo umount /mnt/hd/usr/lib/grub
mint@mint ~ $ sudo umount /mnt/hd

Remaining Issues

After all that, I was able to boot my system into X just fine. Only a few issues remained:

I had fiddled with the touchpad threshold values (in order to make the M4500’s touchpad less insane) and the new values didn’t make any sense on the Thinkpad. I reset them to FingerLow=10 and FingerHigh=20.
Having a different Wifi adapter means all my remembered connections were forgotten. Aparently the saved connections are keyed to the GUID of the WiFi card itself, which on one hand would allow you to have a separate set of connections per card (if you had more than one), but would also mean that just replacing your WiFi device would invalidate all your existing connections, too.
I also had some issues with NVidia (as usual), which for now seem to have been largely dealt with by adding

Section “Device” Identifier “Quadro NVS 140M” Driver “nvidia” Option “NoLogo” “1” Option “RenderAccel” “1” Option “TripleBuffer” “true” Option “MigrationHeuristic” “greedy” Option “UseEvents” “False” EndSection

to /usr/share/X11/xorg.conf.d/20-nvidia.conf. (NoLogo is purely cosmetic; it just disables the NVidia spash screen when X starts up. UseEvents hopefully prevents an issue where the NVidia driver fills up X’s event queue and makes the server hang.)

TeXLive, TLMGR, and Ubuntu

Fri, 13 Nov 2015 00:00:00 -0800

This is another nerd post, but don’t worry, it’s short. I ran into a problem updating my TeXLive installation, and all the solutions I found on the internet were misleading. Note that what follows only applies if you installed your TeXLive from the install script, not via your distribution’s package manager. In the latter case, the error described below is the intended behavior. (But really, in order to get the full use out of TeXLive you shouldn’t be using the repo version, as those tend to be both old and static.)

If you try to update your TeXLive installation on Debian/Ubuntu, you may get an error message saying that it’s “running in user mode” and “can’t initialize TLPDB”. This may surprise you if you, like I, installed TeXLive not via the package manager but from the install script. The causes of this are two fold:

When you run sudo tlmgr by default it does not inherit your path, so if you’ve installed TexLive into /opt or /usr/local it will not be found. Normally this would just result in a “command not found” error except…
If you have also accidentally installed TeXLive through your package manager (e.g., as a dependency of something else), then the tlmgr that sudo will find will be that one. And Debian/Ubuntu TeXLive installations are built to never allow updates to be applied globally, only in user mode (and for good reason: the global set of TeXLive packages is the set that other packages depend on when they have TeXLive as a dependency).

The solution is to purge the package manager copy of TeXLive from your system:

sudo apt-get purge "texlive*"

To run tlmgr via sudo, you’ll either need to explicitly set your path in it:

sudo env "PATH=$PATH" tlmgr update --all

or else add the path to tlmgr to the secure_path setting in /etc/sudoers. The latter is probably more secure, as it only allows running of tlmgr and the associated TeXLive tools from sudo, not everything in your path. (If you want to get all fancy, you could add symlinks to the TeXLive binaries (latex, etc.) into the relevant locations, so that you could then force-install any repo packages that depended on TeXLive, without installing TeXLive.)

Some of the wrong suggestions I saw in my search included:

“Run tlmgr init-usertree to setup your user TeXLive directories”. This won’t work because you almost certainly already have your usertree setup, so it will simply do nothing.
Install xzdec. Apparently the repo copy depends on this, but it’s not listed as a dependency. But the non-repo TeXLive does not suffer from this.
“The behavior is as intended” – For repo-installed copies of TeXLive, yes. But not for the manually-installed version.

Linux Mint on a Dell Precision M4500

Tue, 27 Oct 2015 00:00:00 -0700

A while back I replaced my trustworthy Thinkpad T61 with a Dell Precision M4500. The only problem I had with the Thinkpad was with Nvidia not liking external monitors. (It would restart X when an external monitor was connected, and then the external would be the default monitor. Any attempt to e.g., clone both screens or even to change the default back to the LCD panel would just crash X again.) Since that’s pretty much a deal-breaker for a teacher, I decided to “upgrade” to a Dell Precision. I wanted something in the same “mobile workstation” class, with a good (non-chiclet) keyboard and touchpad. But the Precision has been much more trouble than the Thinkpad ever was (although external monitors do work with reasonable reliability). I’ve been gradually working through the issues as I have time.

Nvidia Sometimes makes Compiz/X Hang or Crash

Sometimes, after logging in, either Compiz will fail to start (leaving me with un-managed windows) or will fail to render anything (giving a black screen) or X will hang completely. The former two cases can be “solved” by killing X (or, equivalently, restarting mdm), but the latter requires a magic-SysRq restart. After reading the usual round of irrelevant forum posts and old bug reports, I’ve tracked down a combination of changes that seem to mostly solve the issue.

Make sure Grub doesn’t switch to the graphical console (or tell the kernel to do the same):

GRUB_CMDLINE_LINUX="nouveau.modeset=0"
GRUB_TERMINAL=console

(Add to /etc/defaults/grub and then run update-grub.) Note that even with this change the kernel still switches to fbcon halfway through the boot process.

Blacklist the Nouveau kernel module: add a line blacklist nouveau to /etc/modprobe.d/blacklist.conf.

Remove the Nouveau X driver (make sure you have the official Nvidia drivers, obviously):

apt-get remove xserver-xorg-video-nouveau

Also note that if you have to restart X, it may leave a zombie Compiz running in the background, taking 100% CPU (which, on an 4-core machine, is not even noticeable). Running top and then doing kill -9 pid will fix that; just make sure you kill the right process.

A related issue seems to be that sometimes the Mate Panel is displayed on screen but not actually mapped, so you can’t click on it. Killing the panel fixes this, but it’s annoying.

Touchpad is Crazy Sensitive

The touchpad was so sensitive I could move the mouse cursor without touching the pad, just by hovering my finger over it. Also, the whole “disable the mouse while typing” thing doesn’t work, which, when combined with the above sensitivity, means that the mouse cursor likes to jump to random locations in my document while I’m typing.

evtest reports some crazy low pressure values: sometimes it will even report movement events with zero pressure. Although the Synaptics driver supports setting min/max thresholds for pressure that will register as clicks, I haven’t yet figured out if it supports just an overall “ignore all events below this pressure” threshold.

The following settings (in .xinitrc) is supposed to fix the sensitivity:

synclient FingerLow=83
synclient FingerHigh=88

But it just has the effect of making the touchpad less sensitive to intentional presses. Accidental touches seem to occur just as often. There doesn’t seem to be a single setting that will set a minimum threshold for pressure to register at all (i.e., as either move or press events).

Note that some people recommend changing the “FingerPress” setting, but that has been removed from newer versions of the driver.

For me, checking the box in the mouse settings that says “Disable while typing” has no effect. I had to add syndaemon -i 0.5s -d to my startup script manually to disable the touchpad while I’m typing.

Dell’s UEFI Implementation is Beyond Broken

Going into the BIOS setup at all completely breaks the normal UEFI boot order. In order to boot back into Linux afterward I have to hit F12 to bring up the boot menu, and manually select the ubuntu EFI option. It doesn’t matter that ubuntu is set as the first, or only, boot entry; if I let the boot go through without my intervention it will hang at a black screen (pre-Grub). The “solution” involves using efibootmgr to change the boot order so that the ubuntu entry is not first in the list, but comes after some inactive entry; in my case, the first entry is that of legacy boot for the very same HD hosting Mint. (The “Modular Bay HD” has Windows 7; it’s replacing the DVDRW drive that would normally be in the expansion bay.) Note that this change cannot be made through BIOS; although the BIOS utility will show what are apparently the same settings, they won’t work. Here’s what the output of efibootmgr -v looks like (with my system currently booting direct into Linux):

BootCurrent: 0005
Timeout: 0 seconds
BootOrder: 0008,0005,0006
Boot0000  Diskette Drive    BIOS(1,0,00)
Boot0001  Internal HDD  BIOS(2,0,00)P0: Crucial_CT256MX100SSD1    .
Boot0002  USB Storage Device    BIOS(5,0,00)USB Storage Device.
Boot0003  CD/DVD/CD-RW Drive    BIOS(3,0,00)
Boot0004  Onboard NIC   BIOS(6,0,00)
Boot0005* ubuntu    HD(1,800,100000,4c57ed8c-a81a-4074-9270-85c4a244fd60)File(\EFI\ubuntu\shimx64.efi)
Boot0006* Modular Bay HDD   BIOS(2,0,00)SATA: ST9250410AS             .
Boot0008  Internal HDD  BIOS(2,0,00)SATA: Crucial_CT256MX100SSD1  .
Boot0009* USB Storage Device    BIOS(5,0,00)
BootFFFA* Internal Shell        Vendor(5990c250-676b-4ff7-8a0d-529319d0b254,)
BootFFFB* Diagnostic Boot       Vendor(5990c250-676b-4ff7-8a0d-529319d0b254,)
BootFFFC* Temporary Boot Menu   Vendor(5990c250-676b-4ff7-8a0d-529319d0b254,)
BootFFFD* Graphic Setup         Vendor(5990c250-676b-4ff7-8a0d-529319d0b254,)
BootFFFE* Text Setup            Vendor(5990c250-676b-4ff7-8a0d-529319d0b254,)

Note that you will have to manually restore this boot order if you do anything that might change or reset it, such as upgrading grub. Merely changing the grub config (update-grub) is fine, however.

I should note that Dell’s implementation is just flaky in general: It seems to largely ignore the boot order specified in EFI, and even though it reports supporting the EFI shell, it will refuse to boot into it. You also cannot use BootNext to tell it to launch (e.g.) BIOS setup on next restart. And it will restore some EFI settings on startup, overwriting any changes you’ve made: you cannot remove the boot entries for physical drives, and they will automatically be added to the BootOrder. (I’ve actually both deleted entries 0006 and 0008, and removed them from the boot order. They were restored and re-added after a restart. There used to be a 0007 entry; it’s missing this boot for some inexplicable reason.)

(Note that you can’t use efibootmgr to modify the UEFI boot settings unless the system was booted in EFI mode. This means that if you started the system in legacy mode (e.g., to run a live CD) you cannot use it to fix your boot settings. But many newer live CD/USB distributions can also be booted in EFI mode; it’s just something to be aware of.)

Performance

Despite being objectively faster (both in terms of clock speed, and core count) and having more memory than my T61, the Precision is noticeably slower. This is with the same model SSD that I had in my T61. I haven’t managed to track this down.