Friday, April 22, 2016

Software and Sharp Tools

In software, we are often tempted to use the most powerful, most flexible, most adaptable tool available. It might be more expensive, more complex, more error-prone, harder-to-debug, etc. But hey, we think, we're smart and careful and we're not afraid to use sharp tools to get the job done.

In the world of electonics, a soldering iron is the ultimate "sharp tool". It allows you to connect anything to anything. But, when was the last time you used a soldering iron to connect your computer to a network cable, USB cable, video cable, power cable, speaker cable, etc? When was the last time you soldered a new video card or RAM board into your computer? The answer is "never". Outside of specialized applications (e.g. research & manufacturing), one never solders components together. At the user level (& even systems assembly), it is always better to have well-defined and unambiguous pluggable interfaces. I cannot accidentally mix up my USB cable and my power cord. I need not worry about getting my video card and memory cards mixed up plugging them into the motherboard. I don't have to worry about reversing the power and data cables to my disk drive.

Similar unique and "type-enforcing" interfaces are used in most other areas where humans can combine simpler elements, including light bulbs & sockets, plumbing pipes & fittings, hardware nuts & bolts, car keys & ignition switches, even CD/DVD disks vs LP, cassette, 8-track (the idea of a unique interface type isn't very new).

While nothing involving humans can ever be guarenteed 100% correct, simply preventing the obvious errors (e.g. power cable <-> USB plug) will eliminate 99.9% of the problem. Shouldn't software take advantage of this simple idea?

Friday, January 8, 2016

Installing PostgreSQL 9.5 on Ubuntu/Kubuntu 15.10 (Wily)

The Ubuntu packages for PostgreSQL 9.5 have been released, but the installation instructions on postgresql.org are a little unclear. After googling to a very helpful blog post (thanks Raony Cardenas!) and some experimentation, here is the succinct version:

> sudo vim /etc/apt/sources.list.d/pgdg.list

add a line for the repositorydeb

> http://apt.postgresql.org/pub/repos/apt/ wily-pgdg main

The mystery bit is the name "wily-pgdg" The first part (wily) should match your Ubuntu release. You can find out the correct name by using:

> lsb_release --all

No LSB modules are available.

Distributor ID: Ubuntu

Description: Ubuntu 15.10

Release: 15.10

Codename: wily

Import the repository signing key, and update the package lists (cut & paste the following 3 lines)

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

sudo apt-get update

sudo apt-get install -y postgresql-9.5

The last line (with "install") is the one that was missing; everything before that just provides the settings to access the postgresql.org Ubuntu package libraries.

At this point, you should be able to see both your old version of postgresql and the new 9.5 version:

> alias d="ls -ldF"

> d /etc/postgresql/*

drwxr-xr-x 3 postgres postgres 4096 May 26 2015 /etc/postgresql/9.4/

drwxr-xr-x 3 postgres postgres 4096 Jan 8 08:49 /etc/postgresql/9.5/

If you still have PostgreSQL 9.4 installed (as above), the new 9.5 will not be on the default port of 5432, but will instead take the next one (5433). You may wish to edit your postgresql.conf to both rearrange the ports (I put the old postgresql on 5439 and the new one on 5432) and also to copy over any modifications to the parameters you have made:

> sudo vim -d /etc/postgresql/*/main/postgresql.conf

You will also want to copy over any changes to pg_hba.conf:

    > sudo vim -d /etc/postgresql/*/main/pg_hba.conf    

At this point, all you need to do is restart both servers, and away you go!

> sudo systemctl restart postgresql

> psql --version

psql (PostgreSQL) 9.5.0

Try to log in:

> psql

psql: FATAL: role "alan" does not exist

Doh! We forgot to create our users for the new install:

> sudo -u postgres createuser alan

> sudo -u postgres createdb alan

Viola! Now it works:

> psql

psql (9.5.0)

Type "help" for help.

alan=> select version();

version

----------------------------------------------------------------------------------------------------------

PostgreSQL 9.5.0 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.2.1-22ubuntu2) 5.2.1 20151010, 64-bit

(1 row)

alan=> \q

Enjoy!!!

Monday, December 28, 2015

Demo to show fail-safe branching & merging in git

The motivation for this post was to demonstrate that git "does the right thing" when merging branches, allowing you to get work done quickly & easily.

Using older tools like CVS and Subverion (SVN), creating and merging branches frequently was difficult and error-prone. Git makes things much easier.

The first thing to keep in mind is that a "branch" in git is really a misnomer, as a git branch is fundamentally different from a branch in SVN or CVS. Suppose we have the following commit history:

A--B--C--D

E--F--G

where the ID numbers are arbitrary "placeholders" for the actual source code. In CVS or SVN, a branch would refer to an entire linear structure, either A-B-C-D or A-E-F-G. However, in git, a "branch" is really just a pointer to one of these code blobs:

A--B--C--D

E--F--G

| |

cc bb

Here, we see 2 "branches" aa and bb (aka "blob pointers"), each of which points to one of the two newest code blobs. Note that we also have another branch cc pointing at blob E, even though it is not one of the newest blobs. As we modify the source code & commit changes, git will add new code blobs to the history graph and will automatically update our blob pointers (aka branches) to point to the newest blob (most of the time).

Onward to the guts of the post!

For the purposes of this post, we will use a few convenient aliases (bash/zsh syntax):

alias d="ls -ldF --color"

alias gits="git status --short"

alias gitb="git branch"

alias gitcam="git commit --all -m'misc' "

alias gitdw="git diff --ignore-all-space --ignore-blank-lines"

alias gitco="git checkout"

alias gitlg='git log -22 --oneline --graph --decorate'

Create an empty directory containing a file and number 5 lines like so:

~/tst > d *

-rw-rw-r-- 1 alan alan 11 Dec 28 15:27 ff.txt

~/tst > cat ff.txt

Create a new git repository (repo) and add/commit the file:

~/tst > git init .

Initialized empty Git repository in /home/alan/tst/.git/

~/tst > gits

?? ff.txt

~/tst > git add ff.txt

~/tst > gits

A ff.txt

~/tst > gitcam

[master (root-commit) 4a43178] misc

1 file changed, 5 insertions(+)

create mode 100644 ff.txt

~/tst >

Create 2 new branches. Note that creating the branches does not mean switching to the branches:

~/tst > git branch aa

~/tst > git branch bb

~/tst > gitb

* master

~/tst >

Switch to branch aa and modify line 1:

~/tst > gitco aa

Switched to branch 'aa'

~/tst > gits

M ff.txt

~/tst > cat ff.txt

1 aa

~/tst > gitcam

[aa 10de35e] misc

1 file changed, 1 insertion(+), 1 deletion(-)

Switch to branch bb and modify line 5

~/tst > gitco bb

Switched to branch 'bb'

~/tst > cat ff.txt

5 bb

~/tst > gitcam

[bb e6d9e33] misc

1 file changed, 1 insertion(+), 1 deletion(-)

We are still on branch bb. Merge in branch aa:

~/tst > git merge aa

Auto-merging ff.txt

Merge made by the 'recursive' strategy.

ff.txt | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

~/tst > cat ff.txt

1 aa

5 bb

Go back to branch aa. Verify it is unchanged:

~/tst > gitco aa

Switched to branch 'aa'

~/tst > cat ff.txt

1 aa

Merge in the changes from branch bb. Note we don't need to worry about getting a "double copy" of the edits to line 1:

~/tst > git merge bb

Updating 10de35e..6111e93

Fast-forward

ff.txt | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

~/tst > cat ff.txt

1 aa

5 bb

Verify our branches. The * character indicates we are still on branch aa (the "aa" part will also show up in green if you are on a color terminal):

~/tst > gitb

* aa

master

So now branches aa & bb are identical. Let's see what happens if we make identical changes on each branch and then try to merge.

Staying on branch aa, modify line 3 to be:

~/tst > cat ff.txt

1 aa

3 xx

5 bb

Commit, then switch to branch bb & make an identical change:

~/tst > gitcam
~/tst > gitco bb

Switched to branch 'bb'

~/tst > gits

M ff.txt

~/tst > cat ff.txt

1 aa

3 xx

5 bb

~/tst > gitcam

[bb 428f3df] misc

1 file changed, 1 insertion(+), 1 deletion(-)

~/tst > gitb

* bb

master

We are still on branch bb. Notice that git sees both branches as being identical:

~/tst > gitdw aa

Try merging in the changes from branch aa:

~/tst > git merge aa

Merge made by the 'recursive' strategy.

~/tst > cat ff.txt

1 aa

3 xx

5 bb

So even though the merge occurred successfully, no changes were made to the file. Git recognized that the changes were already present on each branch and did not "double apply" the change to line 3.

Let's go back to master, then merge in both branches aa & bb:

~/tst > gitco master

Switched to branch 'master'

~/tst > cat ff.txt

No changes are present. Merge in aa first:

~/tst > git merge aa

Updating 4a43178..529c421

Fast-forward

ff.txt | 6 +++---

1 file changed, 3 insertions(+), 3 deletions(-)

~/tst > cat ff.txt

1 aa

3 xx

5 bb

All the changes have been merged into master. Try merging in branch bb now:

~/tst > git merge bb

Updating 529c421..b3d4ecf

Fast-forward

~/tst > cat ff.txt

1 aa

3 xx

5 bb

The "fast-forward" part is how git indicates that no files actually need to be merged; instead, it simply needs to "fast-forward" the branch pointer. We see that the file is unchanged, and that the "merge" was a no-op.

We can see a list of our commits with:

~/tst > git log --oneline

b3d4ecf Merge branch 'aa' into bb

428f3df misc

529c421 misc

6111e93 Merge branch 'aa' into bb

e6d9e33 misc

10de35e misc

4a43178 misc

or a "graphical" picture of the commits with:

~/tst > git log --oneline --graph --decorate

* b3d4ecf (HEAD -> master, bb) Merge branch 'aa' into bb

| * 529c421 (aa) misc

* | 428f3df misc

* 6111e93 Merge branch 'aa' into bb

| * 10de35e misc

* | e6d9e33 misc

* 4a43178 misc

For a better picture of the branches & merges, use the GUI tool gitk:

> gitk

Notice that the last 3 commits have identical contents:

~/tst > gitdw b3d4ecf 529c421

~/tst > gitdw b3d4ecf 428f3df

~/tst > gitdw 529c421 428f3df

since we made identical changes to first branch aa (428f3df), then branch bb (529c421). The merged result was also the same (b3d4ecf) as both of its parents.

Viewing these results, we see that git "does the right thing" when merging multiple branches. It will not double-post identical edits made in different branches. Also, the "master" branch is not special in any way: we could merge branches aa & bb together independently of the master branch.

Enjoy!

Tuesday, May 26, 2015

How to Install NVIDIA Video Card Drivers for Ubuntu/Kubuntu Install or Kernel Upgrade

I have a nice new Linux computer I got from ZaReason this past Christmas, and I decided to install Kubuntu (KDE + Ubuntu) in order to more nearly match my (really slick) XPS 13 Sputnik system from Dell, which comes pre-installed with Ubuntu. I had previously used Fedora 20 on my desktop computer, but wanted to keep the OS on both computers more in sync, and Ubuntu also has better integration with sound, video, peripherals, and other desktop (i.e. non-server) hardware concerns.

The problem is that my nice new EVGA brand video card (with NVIDIA GPU) is not natively supported by Linux, so you have to download and install the drivers from the NVIDIA website. A current driver as of this writing can be found at http://www.nvidia.com/object/unix.html (it might be easier to google "nvidia linux driver download" than to directly navigate the NVIDIA website). Look under the "Linux x86_64" category. I always choose the "Latest Short Lived Branch" since it will have the most bug fixes (e.g. Latest Short Lived Branch version: 349.16). Download this file to your computer, the do a power-off restart.

The power-off part here is important. In configuring the video card, we have a fundamental conflict between the motherboard's built-in video output and the "non-standard" video card output. When you perform a fresh Linux install or upgrade the OS kernel, you are starting from ground zero and the video card will no longer work. Unfortunately, the Ubuntu software update system seems to come up with a semi-major patch every month or two which requires the re-installation of the video drivers. While I could avoid the problem by refusing to perform software updates, I am always eager to get bug fixes and new features in all my other packages, and I just try to minimize the inconvenience of re-installing the video card driver on the new OS kernel.

The problem is that, when the video card quits working, you can't tell what is going on, much less fix it! The following steps are the fastest way I've found to get around these difficulties.

My video card has three (yes, count 'em, three!) video outputs, and I have become addicted to using all three monitors. Yes, I know, it might seem excessive, but trust me: once you've tried three monitors, you'll never (voluntarily) go back to dual monitors again. Of course, the motherboard built-in video output makes it four video output ports in total.

Most monitors (I prefer the excellent 27 inch LG IPS 27MP65) have at least 2 inputs, both HDMI and VGA. The trick is to use all three HDMI outputs from the NVIDIA video card to the three HDMI monitor inputs, and also connect a VGA cable from the motherboard built-in video output to one of the monitors (preferably the least-used monitor as the NVIDIA card sees it).

If only one video input is active, your monitor should normally be able to switch to the active input (HDMI or VGA) port automatically (especially when the monitor powers up). If this sometimes fails, there should be a button labled "Input" or similar that will allow you to manually switch to the desired video input port. Remember this, in case the monitor is displaying the VGA input and what you need to see is on the HDMI input, or vice versa. If you find this problem is occurring, move the VGA cable to a monitor that is not trying to display HDMI. It is much easier if you can see all active inputs simultaneously on different monitors.

When you first install Linux (or after a kernel upgrade), the new kernel will not find the drivers for the NVIDIA video card and the built-in video output will be used. The monitor connected to the built-in VGA output should show the single-monitor display, and you'll be able to install Linux and boot up. At this point, you need to learn a few tricks.

The first trick is knowing that the NVIDIA driver cannot be installed when Linux is in the normal multi-user mode with the graphical X-Windows display active. You must boot into a single-user, text-only mode before installing the driver. On Kubuntu (my favorite) or plain Ubuntu this requires a power-off restart.

When booting, the GRUB bootloader will give you a 10-12 second countdown allowing you to select boot options before it attempts to auto-boot the newest kernel. One of the options is to type the letter "e" in order to Edit the boot options. This is what we need. Type the "e" before the countdown timer expires.

At this point, you should see a screen such as this one:

The important part is in the lower-right-hand corner, where you can see the words "quiet splash" near the end of the next-to-the-last line. The "quiet" part means that status messages to the console during boot are suppressed. The "splash" part means that the OS boots into the multi-user, graphical X-Windows mode. Both of these must be disabled in order to install the NVIDIA video driver.

Using the arrow keys, maneuver to the "quite splash" words, then use the Backspace or Delete key to remove the offending words and replace them with the single word "text". When you are done, the screen should appear like this:

You may now follow the instructions at the bottom of the screen to continue the boot with your modified boot params (e.g. Crtl-X). You should see boot status/progress messages scrolling past on your VGA-port connected monitor. Upon successful boot, the VGA-port monitor will look like this:

Voila! You have booted into single-user text mode. Go ahead & login and navigate to the directory containing the new video driver (typically ~/Downloads).

In this screen shot, you can see that I have three NVIDIA drivers downloaded (I keep forgetting to delete the older two). Note that I don't always download the absolute most recent driver for each OS kernel update (maybe 2-3 times/year I'll download the latest driver from the NVIDIA website). Pick the newest (or only) driver you have, and invoke it as shown with "sudo bash NVIDIA-xxxxyyyyzzz.run":

While the driver is a shell script, I find it easier to invoke it using bash rather than change permissions and invoke it directly. You need the "sudo" part since the video driver installation script must run with root permissions. At this point, the driver will use "ASCII graphics" to complete the installation process. It will stop about a half-dozen times to ask your permission for things. Use the arrow keys to highlight the "Yes" answer, then hit <return> to invoke your selection. Keep doing this until the installation is complete.

At the end of the installation, your VGA screen will be blank, and you'll be back at the command prompt. Here, we need another power-off reboot. Using "sudo poweroff" is a quick & easy way to do that.

Once the system has powered-off, hit the power button and begin a normal boot. After the GRUB timer countdown, you will likely see an "Ubuntu" splash screen for a few seconds on the VGA output. At this time, the computer should recognize your new video driver and switch to the three HDMI outputs of the video card and complete the boot process. The VGA output will be unused from here on.

If your monitor is like mine, it may be confused by having VGA input that is active for 5-10 seconds, but then dies. It keeps looking for the VGA input to come back and does not switch over the the (now active) HDMI input. You can either use the monitor's Input switch to manually switch over to the HDMI input, or perform a quick power-cycle which should cause it to recognize the active HDMI input and ignore the inactive VGA input.

That's all, folks! Happy Linuxing!

Alan Thompson

P.S. For some cool stuff that you wish was in Clojure, you may wish to visit: https://github.com/cloojure/cooljure :)

Saturday, January 3, 2015

Configuration of the LG 27MP65 (aka 27MP65HQ or 27MP65HQ-P) monitor

Configuration of the LG 27MP65 (aka 27MP65HQ or 27MP65HQ-P) monitor

I just purchased an LG 27MP65 monitor and noticed the text quality in the editor didn't seem to match up against a previous, similar LG monitor. As the new monitor was a bit higher grade, this was most surprising.

There are many menu settings that allow customization of the display. After reading the LG manual (http://www.lg.com/us/support-product/lg-27MP65HQ-P) & playing around with them these are the best settings I've found for software development (e.g. lots of text viewing in editors).

- Turn on the monitor to start.

- Press MENU button.
- Press DOWN button (i.e. button under downward triangle) until RESET is highlighted.
    - Press RIGHT button (i.e. button under rightward triangle) once. This will reset all
      settings to factory default (good starting point).

- Press MENU button (left-most button)
- Press DOWN button until NEXT-MENU is highlighted. Press RIGHT button.
    - The first sub-menu is PICTURE (already selected)
      - Leave SHARPNESS at the default 5. Lower values blur text, while higher values
        just wash it out by adding white to colors.
      - BLACK LEVEL should already be at HIGH - leave alone.
      - Always leave OVERSCAN at OFF
      - Press the "return" button (left-then-up arrow) to return to "top" menu.
- Press RIGHT button to select COLOR. Press DOWN to select COLOR RESET on 2nd page.
    Press RIGHT to perform color reset. Since you just did a "master" reset this should
    really be unnecessary.
      - Press the "return" button (left-then-up arrow) to return to "top" menu.
- The remaining 3 top-level items can be ignored. Select the EXIT option.

- Just for fun, press the READER button (2nd from left). There are 3 levels: 1, 2, OFF.
Make sure OFF is selected it should be already).
- Reader mode is used for e-books, etc and is not good for editing software.
- The READER menu will disappear after a few seconds

- Press FUNCTION button (3rd from left).
- The first sub-menu is SUPER ENERGY SAVING (already selected)
    - Press DOWN, then RIGHT to set SUPER ENERGY SAVING to OFF.
    - Press the "return" button (left-then-up arrow) to return to "top" menu.
- Press RIGHT to select PICTURE MODE
    - Press DOWN to select PC/AV Mode.
      - Press RIGHT until it says PC
    - Press DOWN again to the rows of modes
      - Press RIGHT to select a mode. I have found that either TEXT or CINEMA looks best
        for GVim and GMail. Since CINEMA gives slightly better colors for
          syntax-highlighted text, that is my favorite. Your milage may vary....
      - Press the "return" button (left-then-up arrow) to return to "top" menu.
- Press RIGHT to select SUPER RESOLUTION
    - Leave it at OFF. Increasing this value seems to add white to all colors, simply
      making them brighter
- The remaining 2 modes have no effect

- If you are using the DSUB (analog VGA cable) input connection, press the AUTO button
(4th from left). This will perform auto-tuning for the analog VGA input signal.
- You may be happiest using an HDMI input source since it is digital
- NOTE: Since HDMI is digital, ANY cable is just as good as any other. It will either
    work perfectly or you will get no signal at all. So PLEASE, don't spend $100+ on HDMI
    cables. The $5 cables on sale are just as good.

- Just for fun, press the INPUT button (right-most button). The display will blink
momentarily, then verify the input source (e.g. HDMI).

- Note that the first item under the MENU button is the brightness. If you read a lot of
text that is black on a white background (e.g. GMail & similar), you may wish to reduce
the brightness to 70% or some other comfortable value.
- Don't forget that this control is still available if you need to reset the BRIGHTNESS
    to 100% for do more intense work (photo editing, watching videos, etc).

Wednesday, August 27, 2014

Canon Pixma MX922 Scanner - Fedora 20 Linux install

Summary:

Get the Linux drivers from the Canon site
Choose *.rpm.tar.gz format
Untar: tar-xzf rpm.tar.gz

Drivers: You must visit the Canon-Europe site: http://www.canon-europe.com/Support/Consumer_Products/products/Fax__Multifunctionals/InkJet/PIXMA_MX_series/MX924.aspx?type=download&language=EN&os=Linux

Download 2 RPM tarballs for use on Fedora

Installation: TBD

This blog: http://www.ernielevesque.net/its-scangearmp-using-canons-mutli-function-printer-scan-utility-with-linux/

points out that you need to use the command-line util "scangearmp" to invoke Canon's linux scanner utility. I can only assume that "Scan Gear MP" implies "multiple pages"(?). Anyway, the name is not documented anywhere I can find....thank god for this blog and the internet!

Connection: I am just using the USB connection for now; no wireless features enabled yet.

Alternative: I have used VueScan in the past and it worked well (http://www.hamrick.com/). I will later evaluate this as an alternative to the Canon scangearmp utility.

----------------------------------------------------------------------------------------
Original post:

ernie levesque's clues

travel, software and more

it’s scangearmp ! using canon’s mutli-function printer scan utility with linux

I bought a Canon Mx-922 “all-in-one” printer, scanner, fax. copier. I was hoping to use is a printer and an occasional copier but I also wanted to use it to rid myself of the last vestiges of my filed papers. Some of my papers have to be kept as originals, of course. I have to keep deeds and titles but a lot of what I have I mostly likely will never need for anything except maybe a reminder. These papers have no legal value as originals. That was the plan but it ran into some problems at the outset.

The printer itself is easy to set up. It is wireless and you can set it up with WPS or with a pass phrase. You could also set it up with a Ethernet connection. I have a router that allows me to connect a printer but I chose the WPS option and the three computers on my network were installed and working in minutes. I should mention that all if these computers are Linux or Android based.

The scanner was an entirely different story. I expected that the scanner would work like my canon scanner. I bought the canon scanner years ago. It is a single page flat bed scanner and it was plug and play ready the first time I used it. Xsane recognized the scanner over USB immediately. I expected that this new device would work similarly but it did not. I connected it directly to one computer using the USB port but it did not recognize the device.

I was able to find out with some research that Canon had a utility called Scangear. I went to their website but I could not find any Linux software available for the MX-922. A little more research led me to the Canon European site. The utility is located here:

http://www.canon-europe.com/Support/Consumer_Products/products/Fax__Multifunctionals/InkJet/PIXMA_MX_series/MX924.aspx?type=download&page=1

You’ll notice that the device is MX-924 not MX-922 in the URL but the software is for the 920 series. To get the actual download you have to select the download tab and then the operating system (Linux) and the language (English). The offer the complete source library if you want to make your own options but I selected the Debian package.

The package unzips to include an “install.sh” file. I opened a terminal window and went to the directory containing the file and ran it (./install.sh). Everything worked just fine but my computer still did not recognise the scanner. I was very frustrated at this point and I gave up for a day.

On the following day I searched for new information but this time I only looked for the name scangear. I did not reference my printer or Linux. I came across one page where the writer spelled out that scangear is not just a set of drivers, it is also a scan tool and that tool is called “scangearmp”. I gingerly typed the name into a terminal window and was greeted by a info window that said there was no scanner. I couldn’t believe it! Then I noticed that I could “scan” again for scanners. I did this and it found my scanner and poped up a full fledged scanning tool. I was estatic!

Now I’m able to scan multiple pages, on both side, from my computer. The tray hold between 20 and thirty pages and the finished product is a PDF file. All and all it is very useful tool. Why couldn’t they just call it “scangear”? I know that the “mp” at the end probably means “multi-function” but why not just use the name used to document the tool? And, why not put Linux based software on the US site? Why make me find it on the European site?

| JULY 25TH, 2013 | POSTED IN HARDWARE, SOFTWARE |