Dominik Honnef

Writing Go in Emacs

2013-03-11T08:39:35Z

Using the right tools plays a big role in getting your job done efficiently. In the case of programming Go, most of your time will be spent in an editor. It is therefore key that your editor doesn’t get in your way and, optimally, assists you wherever it can.

In this article, I will talk about go-mode for Emacs, its history and features, as well as useful extensions to it.

Even though this article is targeted at Emacs users, parts of it, mainly the feature list, might be of interest to users of other editors as well, to see what’s possible and to adopt some of the ideas. You will want to skip the History and Obtaining it sections though.

History of go-mode

Go has been including a mode for Emacs for a long time now. Until recently, however, it was rather fragile. It regularly messed up indentation or syntax highlighting, for quite some time it could completely lock up your editor and it didn’t feel really integrated in Emacs, lacking support for functions like beginning-of-defun. It also didn’t include a lot of extra features.

All in all, it was better than having to use nano, but not by much.

All that changed around February this year, when I rewrote the mode from scratch. I took special care to build it on top of a solid foundation, making use of Emacs’s parsing facilities, as opposed to the previous version of the mode, which implemented its own parser. Apart from generally bug-free behavior, this also allowed building a lot of functions on top of it.

Obtaining it

There are two ways of obtaining go-mode and both are very similar. The first way is using the one that comes with the Go distribution itself, the second way is to get it from GitHub, where I push all new changes before they find their way into the Go repository.

When using the one that comes bundled with Go, be sure to use the one that comes with tip, the development version of Go. The current release, 1.0.3, includes the old, outdated version of go-mode.

Installation

If using go-mode from Go tip, put both go-mode.el and go-mode-load.el into a directory of your choice, add it to Emacs’s load paths via

(add-to-list 'load-path "/place/where/you/put/it")

and require it via

(require 'go-mode-load)

This will install autoload entries and associate *.go files with go-mode.

If using go-mode from the git repository, the procedure is similar, but requires an additional step to generate the go-mode-load.el file. From within Emacs, run M-x update-file-autoloads, point it at the go-mode.el file and tell it to generate a go-mode-load.el file. From that point on the procedure is the same as before.

Features

For Emacs, a programming mode is often about more than just indentation and syntax highlighting. Integration with the tools of the language is often key, easy navigation of code, even crossing package boundaries, is appreciated and some people even rely on on-the-fly syntax checking.

go-mode, either directly or by using the work of others, offers all of this and more.

We’ll discover how to read documentation, format and navigate code, how to use Go’s own pastebin, how to get syntax checking, autocompletion and snippets.

Reading documentation

Using the godoc function, you can invoke the identically named Go tool from within Emacs and read package documentation in a view-mode buffer. Additional feature: You can tab complete import paths.

Formatting code

Maybe one of the most important tools of Go is gofmt, which automatically formats your code to the one true coding style, used by every Go developer.

In Emacs, there are two ways to use gofmt. One way is to invoke gofmt manually with the identically named function gofmt, which will patch the current buffer according to gofmt. The other way is to use a before-save-hook to run gofmt every time you save a Go buffer. Enabling that hook is as easy as doing

(add-hook 'before-save-hook 'gofmt-before-save)

If you have used this hook before and complained about your cursor jumping around, don’t worry: This has been fixed in the new go-mode. At the very most, your cursor will be placed on the beginning of the current line, if it had to be reformatted.

Managing imports

On a first glance, managing imports in Go doesn’t seem like much of a deal. You either add import statements at the top of your file or you remove them. But when rapidly developing a new application, this can cause a lot of jumping around. Did you just realize that you forgot to import the fmt package? Previously, this meant going to the beginning of the buffer, advancing to the block of import statements (if it existed yet), going to its end and adding your new import. And when you realized you didn’t need it after all, you had to repeat the same procedure to remove it again.

The new go-mode has three functions for working with imports: go-import-add, go-remove-unused-imports and go-goto-imports.

go-import-add

go-import-add, bound to C-c C-a by default, will prompt you for an import path (again supporting tab completion) and insert it in the import block, creating it if necessary. If an import already existed but was commented, it will be uncommented.

If prefixed with C-u, it will ask you for an alias, too. An annoying procedure of moving around and mental context switching has just been reduced to a keystroke.

go-remove-unused-imports

Instead of offering a function for removing a single import, go-mode will detect all unused imports and delete them (or comment them) once you run go-remove-unused-imports. It is not bound to a key by default, but you can bind it yourself if you want to. Personally I have bound it to C-c C-r:

(add-hook 'go-mode-hook (lambda ()
                          (local-set-key (kbd "C-c C-r") 'go-remove-unused-imports)))

go-goto-imports

If you decide you want to look at your imports or edit them manually, go-goto-imports will take you to them automatically, placing your cursor after the last import. It isn’t bound to a key, either, mainly because I couldn’t come up with a good default that didn’t violate Emacs guidelines. But you can bind it manually, just like before:

(add-hook 'go-mode-hook (lambda ()
                          (local-set-key (kbd "C-c i") 'go-goto-imports)))

Navigating code

Also important for efficient programming is quick navigation of code. go-goto-imports was only the tip of the iceberg.

go-mode supports beginning-of-defun (C-M-a) and end-of-defun (C-M-e), two core Emacs functions for navigating between functions. Additionally, functions such as narrow-to-defun and mark-defun rely on these two functions.

If that isn’t enough, you can also use Imenu to jump to specific function or type declarations.

Even if you long for a feature that’s usually only provided by IDEs, or by making use of etags, namely jumping to the declaration of a symbol under your cursor, go-mode got you covered.

go-mode integrates with godef, an amazing little tool written by Roger Peppe. godef is able to parse your code, and the code of other packages, and the code of the Go standard library, and can tell you what exactly the symbol you’re looking at means and where it has been defined.

go-mode makes use of this to provide the two functions godef-describe and godef-jump. godef-describe will tell you what you’re looking at, while godef-jump will take you to its definition. And yes, this works across files, packages and into the standard library, without needing any tags. And it has almost no measurable delay.

godef-describe is bound to C-c C-d and godef-jump is bound to C-c C-j. If you want to, you can bind godef-jump to M-., which is the default key for find-tag, something you might already be using for other programming languages:

(add-hook 'go-mode-hook (lambda ()
                          (local-set-key (kbd \"M-.\") 'godef-jump)))

There are two important notes:

Update (2013-03-19): The godef features in go-mode are now part of the Go repository. As of writing this article, the godef functions haven’t reached the Go repository yet. They are only available in the git version. This will hopefully change in the next couple of days.
You will obviously need to install godef, which is as easy as doing go get code.google.com/p/rog-go/exp/cmd/godef.

Your browser does not support the video tag.

Interacting with the Playground

Go has its own pastebin, called the Playground. When asking for help with code, or sharing solutions to those who need help, it is often expected to use the Playground, especially because it can execute code.

It makes sense that go-mode integrates with the Playground. It offers go-play-buffer and go-play-region to send the current buffer or region to the Playground and store a link in your kill ring.

It can also directly download a paste from the Playground into Emacs, by using go-download-play.

On-the-fly syntax checking

FlyMake is Emacs’s solution to on-the-fly syntax checking. Many programming modes offer support for it (some more easily than others).

In the case of Go, Doug MacEachern wrote goflymake, which consists of a small Go binary and some elisp to integrate it with Emacs. Because Go compiles blazingly fast, using goflymake doesn’t cause any performance penalties. Personally I am letting FlyMake compile my Go buffers every time I insert a newline. More conservative settings would compile after a certain amount of idle time, or when saving the buffer.

Autocompletion

gocode, written by nsf, provides the kind of autocompletion that you might know from IDEs. It provides context-sensitive autocompletion and works across a variety of editors, including Emacs. The project’s README includes detailed installation instructions.

Note: it suggests using auto-complete-mode and mentions company-mode as an alternative. Personally I recommend trying company-mode first. auto-complete-mode isn’t as lightweight and can be harder to configure.

Snippets

Some people like them, some think they’re useless: Snippets. Short words that, upon pressing TAB, turn into longer words, sentences or code constructs. One of the more popular implementations of snippets for Emacs is YASnippet, and it is YASnippet that I wrote a small number of Go snippets for.

Finding unchecked errors with go-errcheck

Last but not least, there’s go-errcheck, offering Emacs integration for errcheck. errcheck is a tool written by Kamil Kisiel for finding and reporting unchecked errors in your Go code.

go-errcheck offers the go-errcheck function, which will run errcheck on the current package and report errors in a compilation buffer. Additionally, it supports setting errcheck flags on a default and per-package basis.

Conclusion

I hope I was able to give you a good overview of the features that you can enjoy while writing Go code in Emacs. If I forgot any cool packages, or if you have suggestions or comments, please leave me a message. Check the contact page to find ways of contacting me.

Go: On implicit type conversions, type identity and a little gotcha

2012-12-05T22:08:35Z

A well known design decision in the Go programming language is the absence of implicit type conversions. An int is always an int and not an int32 and certainly not a float. And a net.IP is not a []byte… or is it?

Knowing only the “no implicit type conversions” principle, the following code should not compile:

package main

import (
    "fmt"
    "net"
)

func main() {
    var foo net.IP
    var bar []byte = []byte{1, 2, 3, 4}
    foo = bar

    fmt.Println(foo)
}

But it does! Now, the definition of net.IP is type IP []byte, but we also know that this should be an entirely new type and not an alias, because Go doesn’t have type aliases, either. So why does this compile and execute properly?

The secret lies in Go’s understanding of type identity and assignability. The following part is of special importance:

A value x is assignable to a variable of type T (“x is assignable to T”) in any of these cases:

[…]

x’s type V and T have identical underlying types and at least one of V or T is not a named type.

And in our case, []byte is not a named type (unlike byte or net.IP, which are), which means that values of type net.IP and []byte can be freely assigned to each other. And of course the same applies to all other identical scenarios, net.IP and []byte are just examples.

The remaining question is: Why did they add this special case to the language specification? Go is not a language of odd exceptions, but at first glance, this exception does seem very odd. The only sensible explanation I came across are function types. Consider the following code:

package main

import "fmt"

type BinaryOp func(int, int) int

func Do(fun BinaryOp, a, b int) int {
    return fun(a, b)
}

func main() {
    result := Do(func(a, b int) int { return a + b },
        1, 2)
    fmt.Println(result)
}

If BinaryOp (a named type) and func(int, int) int (an unnamed type) weren’t identical, writing anonymous functions would be quite a bit uglier.

I don’t know if the initial example, identity of named and unnamed slice types, was intentional or an accident caused by an overly broad specification, but I’d recommend not making use of it and sticking to explicit conversions, even if they’re not necessary. Clarity of code trumps saving a few keystrokes.

Dealing with encodings in Go

2012-04-16T15:31:41Z

One not well documented aspect of Go is how it handles string encodings, or how strings are actually treated internally. This article tries to shed some light on basic aspects of that, to get you started.

It’s all UTF-8, right?

A source of confusion is the fact that Go source code has to be encoded in UTF-8, meaning that string literals, variable/function names etc. most consist solely of UTF-8 code points. This does, however, not mean that strings in Go must only contain valid UTF-8 data. For example, the following string literal is okay to use, even though it does not describe a valid UTF-8 string: "Hello, \x90\xA2\x8A\x45".

In fact, the string datatype is effectively nothing more than an immutable byte array. This also means that using the index operator will return a specific byte, not a rune:

fmt.Printf("0x%x", "世界"[1]) // => 0xb8

Furthermore, the len() function will also return a string’s length in bytes, not runes:

fmt.Println(len("Hello, 世界")) // => 13

But isn’t Go aware of UTF-8?

Even though strings are mere byte arrays, Go does have proper support for UTF-8. For example the range operator will iterate over a string by yielding runes, instead of bytes:

s := "Hello, 世界"
for _, r := range s {
  fmt.Printf("%c ", r) // => H e l l o ,   世 界
}

And by importing the unicode/utf8 package, it is also possible to validate strings, or more importantly, get their rune count (which effectively can be compared to the string’s length as we’d interpret it):

s := "Hello, 世界"
fmt.Println(utf8.RuneCountInString(s)) // => 9
fmt.Println(utf8.ValidString(s))       // => true

And how do I deal with other encodings?

Even though Go has good support for UTF-8 (and minimal support for UTF-16), it has no built-in support for any other encoding. If you have to use other encodings (e.g. when dealing with user input), you have to use third party packages, like for example go-charset. go-charset makes it quite easy to convert between different string encodings.

If you want to further work with input data, the best idea is to first convert it to UTF-8 and then use Go’s built-in features:

package main

import (
  "code.google.com/p/go-charset/charset"
  "fmt"
  "io/ioutil"
  "strings"
  "unicode/utf8"
)
import _ "code.google.com/p/go-charset/data" // include the conversion maps in the binary

func main() {
  s := "Hello, \x90\xA2\x8A\x45" // CP932 encoded version of "Hello, 世界"

  r, _ := charset.NewReader("CP932", strings.NewReader(s)) // convert from CP932 to UTF-8
  s2_, _ := ioutil.ReadAll(r)
  s2 := string(s2_)
  fmt.Println(s2)                         // => Hello, 世界
  fmt.Println(len(s2))                    // => 13
  fmt.Println(utf8.RuneCountInString(s2)) // => 9
  fmt.Println(utf8.ValidString(s2))       // => true
}

Conclusion

Even though this article is rather short, I hope that it was able to convey the following main points:

Strings in Go are mere byte arrays
There is built-in functionality for dealing with UTF-8
Any other encoding is best converted to UTF-8 first

For further reading, I recommend a discussion on the Go mailing list, which also sheds light on why strings behave the way they do (hint: it’s mostly about performance, and freedom).

Breaking backwards compatibility in good conscience

2012-03-28T10:57:31Z

A couple of days ago I released Cinch 2.0 – version 2.0 of the most-used IRC library written in Ruby. The purpose of this article, however, is not to talk about all the new features and improvements in Cinch (check the changelog if you’re curious) but why it breaks backwards compatibility and how that’s not a bad thing.

The Ruby community basically has two main camps: Those who think that breaking backwards compatibility is no problem, as long as your version numbers signalise it (c.f. Semantic Versioning), and those who think that breaking backwards compatibility is proof of bad API design and should be avoided at all costs.

Loren Segal, creator of YARD, belongs to the latter camp. Notably, the current version of YARD (0.7.5 as of writing) is still compatible with the earliest releases. According to him, he still uses the same plugins and templates that he wrote for version 0.2.x, released roughly 4 years ago.

He says that this is possible because he put a lot of thought into the API, planning before writing code. While I agree that maintaining backwards compatibility is a noble cause, and that planning can help, I don’t think that everything can be planned perfectly. I think that Loren accomplished a great feat by planning the API in such a solid way that it worked out for him, but it might not for others, and it might not be a sign of bad will.

I myself put quite some thought into the API of Cinch before releasing version 1.0 – I tried to consider as many scenarios as possible, planning ahead and avoiding future conflicts. And I failed. Cinch was my first IRC library with focus on writing multithreaded bots, and while I tried to get everything right, I overlooked many little details. Details that only became apparent when users came to me, pointing out problems they were having.

While I could’ve maintained some parts of the flawed API in version 2.0, other parts had to be removed completely because they were plain broken and unfixable. And I didn’t want to maintain the flawed parts, either. Why? Because they added unnecessary complexity to the code base of Cinch, and because on the long run it won’t matter. Cinch is now 1.5 years old and it will be around for much longer. In a couple of years, nobody will care that the API changed. In fact it will probably not take more than a couple of months.

My point is that it’s a good idea to try and avoid breaking the API, but only as long as it makes sense and as long as it doesn’t have a negative effect on your code, or the rest of the API, in the long run. And if you’ve made a mistake while designing the API, it is better to admit the mistake and fix it than to live with it for the rest of your project’s life.

Everyone who has taken a look at the code base of YARD knows that it indeed has been designed very well, but I think this is an exceptional case and not the norm and it shouldn’t be assumed that everyone can achieve the same. Maybe it was even a bit of luck that it worked out so well.

Of course this isn’t a free ticket for breaking APIs every second release, either. If you have to constantly break backwards compatibility, maybe you should put at least some thought into design first.

Linux RAID + dm-crypt: Poor write performance

2011-10-28T23:41:47Z

I’ve been running a RAID 6 with four active disks for quite some time now, and performance always seemed adequate, achieving about 230MB/s read and 90MB/s write (obviously in rather artifical tests using dd; actual file operations might vary).

Now it occured to me, that encrypting my data might actually be a good idea, so I looked into cryptsetup for adding encryption on top of my RAID, using LUKS. Now, anyone familiar with RAIDs should know that alignment is key to proper performance. cryptsetup offers an option called --align-payload to align data at a multiple of 512 bytes. Based on the chunk size I am using (512KB) and the number of disks, 2048 seemed to be the right value for the setting.

Weirdly enough, though, it didn’t matter what value I used: 2048, 1024, none at all or completely arbitrary values: Write performance would always suffer and be around 30MB/s, with my CPU being almost idle. I spent hours trying to figure out what I might’ve been doing wrong, why the option wasn’t having any effect and so on, until I found an aligning-unrelated setting called stripe_cache_size, which defaults to 256 on many systems, mine including. After setting that option to a higher value of 8192, my write speeds rocketed up to 190MB/s for both plain md0 as well as the encrypted device.

The option can be set by simply writing a value to /sys/block/<your_md_device>/md/stripe_cache_size, so for example: echo 8192 > /sys/block/md0/md/stripe_cache_size.

Please be aware that the setting describes the number of pages to cache, so a higher value will consume more memory.

Also note that I did not rely on the speed output of dd but instead used nmon, to avoid getting higher speeds caused by buffers. In fact, the system will continue writing data while dd already thinks it is done.

Electronics: My first experiences with various shops

2011-08-25T14:53:02Z

This is part three of a series of me describing my new experiences with electronics.

First of all, sorry for the late third article, but ever since I received my Arduino I was really busy playing around with it.

Quite a bit has happened since my last article and I will try to catch up as quickly as possible. One thing that happened is that I had to order quite a bit more tools and components than I initially thought.

In this article I will concentrate on my experiences with the various shops I’ve ordered from so far and how they performed.

bausteln.de

I ordered my Arduino starter kit from bausteln and they were quick to send me a package. Everything was well packaged, nothing was missing and they even included a neat little plastic box for storage.

There was, however, a little problem: The included breadboard is about half the size of the one depicted in images. Now, I know that relying on images is my fault, and the article description didn’t include any measurements, either, so I really shouldn’t have expected a breadboard of a specific size.

But my problem isn’t really with the small breadboard (even twice the size would still be too small for any serious work) but the fact that I’ve tried to contact the owner of the shop (it’s a single person) and never got a reply. I’ve also tried to write a product review to warn potential new customers, but said review has to be accepted by the shop owner and he never did.

I do not know if he simply doesn’t care or just missed the messages, but I do not feel comfortable doing business with someone who cannot even reply to emails or monitor his shop.

Futurlec

Packages from Hong Kong take a long time… Three weeks to be exact, at least if you live in Germany. But obviously Futurlec is not to blame nor did I expect any faster shipping.

I was, however, honestly surprised about their quality. Everything was packaged well, components were new, nothing was bent, nothing was broken and nothing was missing.

As you might remember, I ordered a couple of “value packs” from Futurlec. Basically bags full of discrete components of different sizes/values. What I expected was one big bag per component type, full of stuff, unsorted and with bent legs. I was surprised to see that each value had its very own small bag, properly labelled as well.

Should I ever have to buy components in bulk again, I will definitely consider Futurlec a second time.

Reichelt

Reichelt is a German electronics store, selling everything from components to tools and gadgets. Unlike other German competitors like Conrad, Reichelt has cheap prices and doesn’t completely ignore the hobbyist/maker market.

So far I made two purchases, mostly containing soldering equipment, but also a few ICs and more importantly a big breadboard. Both packages arrived fast (within 1-2 days).

Everything was packaged well, everything worked properly, I am happy.

Pollin

The 4th shop I ordered from is Pollin, another German store. They’re kind of an electronics store, but they concentrate mostly on buying and selling remaining stock and the like. Sometimes they sell random LCD screens without any documentation, or even old Russian ICs.

I ordered 20 little plastic boxes for storing my components, 0.85€ per piece.

Shipping was moderately fast (3 days), everything was packaged well etc. The boxes aren’t of best quality, but hey, I paid 85ct per box and it has a reason they’re that cheap. Render me satisfied.

Electronics: Starting small

2011-07-26T17:57:50Z

This is part two of a series of me describing my new experiences with electronics.

As I said in my last post, I want to start with simple hardware. The reasons for this should be pretty obvious: neither do I have a concrete idea for a project, nor am I certain that I’ll stay interested for long enough. Consequently, I don’t want to spend too much money.

A rough limit I have set for myself is about 60€.

Disregard Females, Acquire Hardware

The first step is to find a platform to work with, where the following aspects are the most important ones to me:

the developing environment has to be compatible with Linux
it has to be affordable
it has to be solder-free
it mustn’t be too limited

Friends quickly pointed in the direction of Arduino, an open-source platform.

What is Arduino?

Arduino is both a community and a hardware platform. The idea of Arduino is to provide an open platform for average people, especially artists (with a wide meaning of artist).

Open, in this context, means that the circuit design as well as the source code are freely available to anyone. An implication of this is that anyone can design (and sell) his own Arduino. A nice side-effect is that no single entity can control the price and thus Arduinos are very affordable, starting at 20€ for the basic ones.

What Arduino to get?

There’s not “one Arduino” that you can buy, but different designs for different tasks. There are Arduinos especially made for sewing into clothes, extremely small ones for small devices and a variety of palm-sized ones for prototyping (which is what we’re after).

The current “revisions” of the main prototyping boards are called “Arduino Uno” and “Arduino Mega 2560”, where the only difference is that the Mega has more pins, more memory and is more expensive.

The Arduino Uno can be bought for about 20-30€ (depending on where you live) and the Arduino Mega for about 40-60€.

Considering that I want to start small, the Arduino Uno is the perfect choice, especially because I still have to get more components to actually be able to do something with it.

Starter kits

Luckily, a lot of shops sell starter kits consisting of an Arduino Uno and some basic components. The most promising one I found is the “SparkFun Inventor’s Kit”, which comes with an Arduino, a lot of sensors and some actuators, for 65€. For more information on the exact list of parts, click the link.

Unfortunately, shipping to Germany is rather expensive, and even though some German shops sell the very same kit, they sell it for more than 65€, so that exceeds my limit, especially the one imposed by my bank account.

I did, however, find a comparable kit at a German shop called “bausteln” for only 59€. It comes with nearly the exact same components, just lacking two of the sensors and a DC motor. Since I already own multiple DC motors, and the missing sensors can easily be bought separately later, I chose to buy that kit.

Buying in bulk

I never realised how cheap resistors, transistors and diodes can be if bought in bulk, at least if buying at shops that specialise on that kind of selling, like Futurlec does.

Beside all the single components they sell, they also sell bags full of them.

$2.95 for 300 resistors? Bought! $2.95 for ceramic capacitors? Bought! $5.95 for Linear ICs? You guessed it, bought! $4.95 for 100 LEDs? Nope, I already have plenty of LEDs, thanks anyway. And so it goes on for most of the “value packs” Futurlec sells. Those are certainly a great way to get you started. For about $25 plus shipping (which is also extremely affordable) you get bags full of everything you’ll need as a starter.

My shopping list

And this is what I eventually ended up buying:

Arduino Einsteigerset – $72
Value packs
- 300 1/4W resistors – $2.95
- 200 ceramic capacitors – $2.95
- 100 electrolytic capacitors – $3.95
- 50 Linear ICs – $5.95
- 100 diodes – $2.95
- 100 transistors – $4.95
Red bargraph – $0.30
Jumper wires – $4.90

In total that’s $100.9 or 70.19€, plus shipping. That’s very close to my limit of 60€ so I am satisfied.

More articles coming once I received my packages.

Electronics: Introducing myself to the world of electronics

2011-07-25T17:29:30Z

This is part one of a series of me describing my new experiences with electronics.

Introduction

I’ve never really been interested in electronics (ie. circuits, microcontrollers etc). Sure, I understand, on a basic level, how most of the gadgets and computers that I own work, but I wouldn’t be able to build anything myself or explain what e.g. a diode is used for.

The closest I ever got to working with electronics was owning a kit for children, consisting of a couple transistors, resistors, capacitors and LEDs. My interest didn’t last long though, especially because I cooked one of the transistors early on, rendering me unable to build most of the circuits from the manual.

Being a geek, however, I am always interested in acquiring new skills that somehow relate to computers and electronics are just a logical extension (or rather simplification) of computers. Furthermore I found the following reasons to learn more about electronics:

Parts of it might be considered common knowledge among my circles
It might be a nice skillset for my CV, especially the microcontroller aspect
My studies at university will require me to work with microcontrollers, anyway
I heard it is supposed to be a fun experience!

So what am I going to do now? I’ll look for an affordable microcontroller and components needed for basic circuits and go from there. More on that in my next articles.

SketchUp Plugin: Selection Manager

2011-07-13T15:00:33Z

Disappointed with SelectionMemory and not willing to pay $5 for SelectionMemory2 (not mentioning the bad interface it has), I went to write my own plugin for saving and restoring selections.

The main benefits compared to other solutions are:

It is free
It has no artificial limit with regard to how many selections can be saved
It is driven via the context menu
Selections can be saved with the model and won’t be lost when closing SketchUp.

For the readme as well as the plugin itself, check the git repository at GitHub.

Don't Abbreviate "Thanks," "Sorry" or "Please"

2011-07-08T10:19:37Z

Author: Matti Virkkunen
Editor: Ludvig Ericson

When you say thanks to somebody, you’re expressing gratitude. You’re telling the person that you really appreciate what he/she did for you. If somebody has given you something without wanting anything in return (and even if they did want something in return, it never hurts to be polite) you really should be grateful. The best way to mess this up is to say “thx” or “10x” or “ty” or something like that. I can’t believe that somebody is really grateful when he doesn’t even have time to type “thanks” but instead abbreviates it to “thx” or something to save himself from typing–get this–three extra letters.

Another thing is saying “sorry.” It’s often said that “sorry” is the most important and at the same time the most difficult word to say. If somebody doesn’t bother to type the whole word “sorry” but instead uses “sry” or “soz” etc, I can’t really believe that he’s truly sorry. It’s more like “Okay, whatever, I’ll just settle this huge mistake I’ve made by saying ‘sry.’”

And the third word, “please,” shouldn’t be abbreviated either. It’s a bit like “thanks” in advance, it means approximately “I’ll appreciate if you help me with my problem here.” It’ll send out a message saying that you’re a nice enough person and will hopefully increase your chances of getting help. Don’t mess this great opportunity up by using “plz” or “pls” or something similar.

I don’t really mind people abbreviating other words although abbreviating just to save the trouble of typing two letters is dumb. I guess abbreviating words like that is fine when you’re talking with people you know really well, and know they won’t mind, but not on some support IRC channel or forum where people donate their time to the community to help others for free. Just don’t abbreviate “thanks,” “sorry” and “please” because they’re meant to express “strong” things, gratitude and regret.

Thank you for reading this.

Matti “Lumpio-“ Virkkunen

[editor’s note: Some text removed. The following paragraphs are whole additions, and do not in any way represent what the original author wrote or intended to write.]

Using aforementioned abbreviations in the purpose of portraying the typical user who would in honesty use them are excused, this is called being ironic or sometimes caricature.

Applying aforementioned thinking on all abbreviations (for example “e.g.”) does simply not make sense. Please note that the point in not abbreviating words that bear a major weight is that a full word shows that you have cared enough to type the whole word out, thereby showing respect and/or engagement in a given task. Had you written out “e.g.” as “exempli gratia,” one would perceive it as emphasis, since such abbreviation is rarely, if ever, written out.

That said, abbreviating too much isn’t very nice either. Most readers would subconsciously or consciously have to reinterpret one sentence or even one word a few times, so while it might take less time to write in an overly abbreviated manner, it takes twice the time for some people to read said text, resulting in unwanted emotions for the reader. Don’t abbreviate too often, when in doubt, go with the expanded version.

[editor’s note: This is a heavily edited version of Matti Virkkunen’s original work (available over HTTP at http://lumpio.no-ip.com/dont-use-thx-sry-plz.txt) and perhaps can not be legitimately said to have been authored wholly by said person.]

rsemantic – Latent Semantic Analysis for Ruby

2011-05-01T14:39:47Z

Introduction

Some time ago, I started reading Where Good Ideas Come From: The Natural History of Innovation and one thing it mentions is an application called “DEVONthink”. DEVONthink is an application you’d use for storing all kind of documents (RSS feeds, emails, notes, articles) and the by far most interesting feature is its ability to recommend similar documents, without asking the user to provide categories/tags or summaries by himself. Instead, DEVONthink automatically analyses all your documents and calculates the similarity.

The really interesting part here is that DEVONthink is able to tell if different words have similar meanings, e.g. “canine” and “dog”. That means that if you’re reading a new article which talks mostly about dogs, DEVONthink will be able to suggest an article about canines. This isn’t done by a fixed list of words with similar meanings, but again by analysing all documents and checking which words get used together often.

Unfortunately, DEVONthink is only available for OS X and not for Linux. Besides that, it works with a centralised database, while I believe in the beauty of plain text files. That is why I began researching how DEVONthink is able to successfully find similar documents and I quickly found Latent Semantic Analysis, short LSA.

Latent Semantic Analysis

Now, I do not want to explain LSA in detail, but summarized it works as follows: First, a term-document matrix gets constructed, storing which terms occur how often in which documents. Such a matrix usually is quite sparse, meaning that it is mostly populated with zeros. LSA then decomposes the matrix into three parts (this is known as Singular Value Decomposition, SVD), reduces the singular values to the K biggest values and then recomposes the matrix. This effectively reduces the dimensions of the vector space, putting similar words closer together.

For a detailed (and mathematically accurate) description, check out the Wikipedia article about LSA.

LSA in Ruby

Trying to avoid having to do the math by myself, I looked for a ready library for Ruby that’d offer LSA. I found rsemantic, which by itself looked promising, not only offering LSA but also tf-idf (a way to give common words negative scores).

The only problem with rsemantic was, that it depends on linalg, another Ruby library, which provides binding to LAPACK, a Fortran library for linea algebra. Now the problem with linalg is that it doesn’t compile on modern systems, at all. Apparently it requires gcc 3.x, and the maintainer has seemingly abandonned the project quite some time ago.

Looking for an alternative to either rsemantic (which wasn’t actively developed anymore, either) or linalg, I found the GNU Science Library (GSL), for which someone wrote working Ruby bindings.

The next step was obvious: port rsemantic to GSL. At the same time, I was able to improve other parts of rsemantic, cleaning up the whole code base and drastically improving the performance, to a point where the only time consuming process is doing the SVD itself.

The future of rsemantic

Luckily, the developer of rsemantic is still interested in the project (and even actively using it, according to him). He therefore merged my changes and added me as a contributor to the git repository at GitHub, allowing me to further work on the library.

A new version of the gem will be released as soon as both of us are happy with all the recent changes.

Furthermore, I am working on an improved API for rsemantic. While currently it is pretty low-level (and not that object-oriented), I am working on a far more high-level API. Following are two examples, first for the new API, then for the old one.

New API

require "semantic"
require "pp"

corpus = Semantic::Corpus.new([], :transforms => [:LSA])

# The directory /tmp/some_texts/ contains Wikipedia articles about
# various animals (beagle.txt canidae.txt cat.txt dog.txt gecko.txt
# german_shepherd.txt reptile.txt salamander.txt)
Dir.glob("/tmp/some_texts/*.txt").each do |file|
  # Here we add each article to the corpus (collection of documents).
  # `:name` is an attribute we're assigning to the document. There is
  # no limit in the names and amount of attributes you can attach to
  # each document.
  corpus << Semantic::Document.new(open(file), :name => File.basename(file))
end
# Build the index, Depending on the CPU, available memory and most
# importantly the size of the corpus, this can take some time.
corpus.build_index

# Compares the article about dogs to all other articles and returns a
# score for each article, representing the similarity.

# In the example output we can see, that rsemantic correctly says that
# "dog.txt" is by far the most related to
# {german_shepherd,beagle,canidae}.txt, and not to e.g. reptiles.
pp corpus.documents.find { |d| d[:name] == "dog.txt" }.related
# {#<Semantic::Corpus 8 documents, @options={:transforms=>[:LSA]}>=>
#   [#<Semantic::SearchResult:0x0000000283db30
#     @document=#<Semantic::Document @attributes={:name=>"gecko.txt"}>,
#     @score=0.05935175420757509>,
#    #<Semantic::SearchResult:0x0000000283dbd0
#     @document=#<Semantic::Document @attributes={:name=>"reptile.txt"}>,
#     @score=0.09322815190241214>,
#    #<Semantic::SearchResult:0x0000000283db08
#     @document=#<Semantic::Document @attributes={:name=>"cat.txt"}>,
#     @score=0.14170011833185264>,
#    #<Semantic::SearchResult:0x0000000283db80
#     @document=#<Semantic::Document @attributes={:name=>"salamander.txt"}>,
#     @score=0.17269285169838045>,
#    #<Semantic::SearchResult:0x0000000283db58
#     @document=#<Semantic::Document @attributes={:name=>"canidae.txt"}>,
#     @score=0.2107975969277189>,
#    #<Semantic::SearchResult:0x0000000283dba8
#     @document=#<Semantic::Document @attributes={:name=>"beagle.txt"}>,
#     @score=0.4041096720670102>,
#    #<Semantic::SearchResult:0x0000000283dbf8
#     @document=#<Semantic::Document @attributes={:name=>"german_shepherd.txt"}>,
#     @score=0.508139138880475>,
#    #<Semantic::SearchResult:0x0000000283dc20
#     @document=#<Semantic::Document @attributes={:name=>"dog.txt"}>,
#     @score=1.0000000000000688>]}

Old API

require "semantic"
require "pp"

corpus = Semantic::Corpus.new([], :transforms => [:LSA])

documents      = []
document_names = []

Dir.glob("/tmp/some_texts/*.txt").each do |file|
  documents      << open(file).read
  document_names << File.basename(file)
end

# This automatically builds the index for all documents
search = Semantic::Search.new(documents, :transforms => [:LSA])

document = document_names.index("dog.txt")

h = {}
search.related(document).each_with_index do |score, index|
  h[document_names[index]] = score
end


# The resulting scores are the same, but we have to manually keep
# track of file names, indices, and the scores are not sorted, either.
pp h
# {"dog.txt"=>1.0000000000000688,
#  "german_shepherd.txt"=>0.508139138880475,
#  "reptile.txt"=>0.09322815190241214,
#  "beagle.txt"=>0.4041096720670102,
#  "salamander.txt"=>0.17269285169838045,
#  "canidae.txt"=>0.2107975969277189,
#  "gecko.txt"=>0.05935175420757509,
#  "cat.txt"=>0.14170011833185264}

Why You Should Try tmux Instead of screen

2010-10-27T01:42:05Z

What screen is

If you’re reading this article, you probably already know what GNU Screen is: the most well-known terminal multiplexer in the world of Unix. If that doesn’t ring any bells, I recommend reading the Wikipedia article about GNU Screen before continuing with this article.

An Alternative

There are, however, also some other terminal multiplexers besides GNU screen, most notably tmux. While technically screen and tmux serve the same purpose – multiplexing terminals – their functionality and quality differ drastically.

The History of screen And tmux

Development of screen started in 1987; additions and fixes have been contributed ever since. On the one hand, that means that screen is a solid and proven piece of software; On the other hand, though, it also means that the codebase isn’t the cleanest one nor easy to understand. As a result of this, new additions are rare and development mostly concentrates on fixing existing bugs.

Tmux, on the other side, saw the light of day in late 2007. And because the developer was able to learn from screen’s mistakes while at the same time picking up existing and field-tested features, tmux has a solid and extensible code base – and it is actively developed by a very responsive developer.

Actual New Features

So, tmux has a better codebase and has (nearly, more on that later) all the features screen has – but chances are high that you are not interested in the code at all, so why should you care? Because tmux has a lot of features that screen has always been missing.

A Sensible Configuration Format

If there was only one shortcoming of screen that you were allowed to name, which one would it be? Right, the format of the configuration files…

Let’s look at a prominent example: a hardstatus (called “status line” in tmux) that contains the computer’s hostname, a list of all windows, the load average and a clock:

hardstatus alwayslastline '%{gk}[ %{G}%H %{g}][%= %{wk}%?%-Lw%?%{=b kR}(%{W}%n*%f%t%?(%u)%?%{=b kR})%{= kw}%?%+Lw%?%?%= %{y}][%{G} %l%{y}] %{y}[%{G} %m/%d %c %{y}]%{W}'

If I didn’t tell you what that option did, were you able to understand it, or more importantly, able to write it yourself? No, you probably wouldn’t. (If you, in fact, are able to: congratulations, and sincere condolences).

So, how would that look like in tmux? Here’s how:

set -g status-left ' #[fg=green,bold]#H#[default]'
set -g status-right '#[fg=yellow,bold]#(cut -d " " -f 1-4 /proc/loadavg)#[default] #[fg=cyan,bold]%Y-%m-%d %H:%M:%S#[default] '

Okay, so it’s easy to see where colors are being set (and yes, they do differ from the screen example, just ignore that. It would look the same for other colors, too), it’s kind of obvious what “#H” means (and the man pages of tmux have a complete list of all available codes), we instantly can see how the load average is displayed, and the clock format string should be obvious.

But wait, where’s the list of windows? That’s a built-in feature of tmux and thus doesn’t have to be set explicitly.

Of course there are a lot of more options, like the following one, which loads the emacs keymap:

set-window-option -g mode-keys emacs

Another major aspect of tmux’s config format is that it is basically a batch script: it is a simple list of commands that will be executed when tmux is started – they’re the same commands that you use in keybinds, the command prompt and shell scripts (more on that in a minute). If you want to always create two windows when a session is created, you can put this into your configuration:

new-window -n some_name
new-window -n another_name

Scriptable

Another major aspect of tmux is that nearly all commands can be invoked from the command line, allowing complex scripts controlling tmux sessions. tmux new-window, for example, would create a new window. Splitting, moving and resizing windows, setting options and a lot more actions are possible as well. Even the status bar can dynamically be changed.

Emacs And Vi Keymaps

tmux supports two different keymaps, imitating either Emacs or vi. Those keymaps will be used in copy mode and the command prompt.

Mouse Support

Yeah, power users will smile at this feature, but novices might actually like it: panes can be selected using the mouse, and in copy mode, the mouse wheel is supported as well.

Menus

Nice to have, tmux has menus for selecting windows and sessions.

Multiple buffers

Much like the kill ring in Emacs, tmux can keep more than one selection/piece of copied text in memory.

BSD License

Unlike GNU Screen, tmux is licensed under the simplified BSD license. Obviously this will please everyone who dislikes the GPL – but there’s even a real benefit from the choice in license: tmux is part of OpenBSD since version 4.6.

Differences to screen

Server Model

One speciality of tmux is that when creating your first session, it spawns a server which manages said session and all future sessions. In fact it is also possible to change the current session on the fly, using commands or even a fancy navigatable menu.

With tmux, it is not uncommon to have a set of different sessions (one for work, one for server management, and so on) and to quickly switch between them, effectively creating another level of grouping of windows.

Sessions can also be grouped, which means that they share the same set of windows but have independent options and allow multiple users to work with the same windows – much like screen -x, just a tad more flexible.

Window Splitting

This one is probably the most confusing aspect of tmux, to people coming from screen. While splitting in screen means displaying multiple windows at once, in tmux it means splitting a window into multiple panes. It is, however, possible to take a pane and convert it into a window and vice versa. It is, however, not possible to display a window in a pane, without converting it to a pane.

Tmux does, however, natively support both horizontal and vertical splits, without the need for a custom patch, as is the case with screen.

Additionally, tmux has presets for pane arrangements, eliminating the need for arranging splits by hand.

Missing Features

As mentioned earlier, there is in fact a feature missing in tmux: built-in serial and telnet support, while screen is able to directly connect to for example a serial null modem. Even more so, tmux will probably never include that feature, because the developer considers it bloat and says, that other, specialized tools should be used for that task instead.

Final words

I hope that this article is giving you enough reasons to at least try tmux. Of course there might be users who are just happy with screen or even users who need its support for serial consoles, but most of you should be able to at least give it a try. I can assure you that you will not regret it.

I also recommend reading the man pages of tmux, which are written in a very clear and readable way, and just playing with some of the options.

Newton and Cinch – One awesome framework

2010-08-03T23:45:20Z

A lot of time (3 months, actually) has passed since I’ve last blogged about Ruby and available IRC frameworks for writing bots. Back then I planned to fork Isaac or Cinch, in order to add proper abstraction and object-orientation to them. Well, I did fork Isaac, realized that it was, in all honesty, quite a mess and instead wrote my own framework, mostly from scratch. I did, however, call it “Newton”, completly ignoring the fact that Isaac was named after Isaac Hayes and not Isaac Newton. I still thought it was a suiting name for what was supposed to be a fork.

Well, in my obviously biased opinion, Newton is the finest Ruby framework for writing IRC bots. It has it all:

a nice DSL
proper debug output
threads
support for all relevant parts of the various IRC RFCs, including support for ISUPPORT
support for plugins (in a separate branch)

There is, however, one thing that Newton is missing: Users. Shockingly enough (*cough*), good code (remember? biased opinion…) and features alone won’t attract a lot of users. Good advertisment does, though, and since you probably have never heard of Newton before, in all those months, there doesn’t seem to be any good advertisment to be going on.

This is where Cinch is coming into play. Cinch has remarkably many users, even though I think that not even its developer knows how that’s possible. Anyway, influenced by Newton (and a funserious rivalry between Cinch and Newton), the developer of Cinch decided to adapt “some” of my ideas, including more abstraction, tracking changes in state (joining/leaving users etc.) and developing a plugin system (admittedly, he did it first, Newton stole that idea). Threading was also on his to-do list.

And what’s the second most obvious thing to do, if two projects share the same ideals and both desire attributes of the other project? Right, consider merging the two projects (the most obvious thing is starting a huge flame war, of course).

And this is why there won’t be any new Newton releases. Instead, most of the current codebase of Cinch will be replaced by Newton, while some features of Cinch (its pattern matcher, mostly) will be backported to Newton. Yes, this will mean that all existing Cinch bots will not be compatible to the upcoming version, but on the other side, new and updated bots will be able to profit from a lot of new awesome features, which will be documented in a later post, and of course an official README. The current work on the merge can be seen in the merge branch, which will become the master branch in a few weeks.

Lastly, for real facts on why the merge is good for Cinch and Newton:

Good for Cinch, because…	Good for Newton, because…
Awesome new features (threading, OO, plugins)	A new name, known by a lot of people
Two developers instead of one	Reworked documentation, which will be friendlier to newbies
	Two developers instead of one

That’s it for this article, stay tuned…

Are lightweight IRC frameworks too lightweight?

2010-05-01T19:07:07Z

Recently I’ve been looking for a small framework for writing IRC bots in Ruby. Admittedly, I have been successfully running a bot based on Butler for quite some time now, but I have always been looking for a more lightweight solution, which does not come with a predefined plugin or authentication structure.

Now there are probably three major players in the field of IRC bots, namely Autumn, Isaac and just recently Cinch.

I won’t further examine Autumn here, as it is to IRC bots what Rails is to web development: Anything but lightweight.

Isaac and Cinch are quite similar: They both feature a small DSL for quickly developing bots, while Cinch puts more emphasis on the idea of plugins (Isaac is more about parsing messages in general) and its own pattern language to match input.

But there is another thing they have in common: No proper object-oriented interface… Instead of passing around channel and user objects, all the developer sees are strings. This also means that instead of doing something as intuitive as

some_channel.part
# or
some_channel.kick(user, "some reason")
# or
some_channel.topic # retrieve the topic
some_channel.topic = "new topic" # set a new topic
# or
some_user.hostname

one will have to fall back to stuff like this (the examples are based on Cinch but will be similar for Isaac):

bot.part(some_channel) # now this still makes sense
bot.kick(some_channel, user, "some reason") # not too nice anymore
bot.topic(some_channel) # plain wrong
bot.topic(some_channel, "new topic") # plain wrong
# there simply is no method of retrieving a user's hostname,
# except from doing a WHOIS and parsing the output yourself

Of course one might argue that those frameworks/DSLs are lightweight after all and that the user could write helper methods if he really wanted to have a rich interface, but that would also mean that I had to use those helper methods every other line, just to deal with proper objects.

It is quite questionable if “lightweight” is even allowed to mean “not properly abstracted” or if abstraction is something that can be taken for granted if working with Ruby.

Either way, as nice as the DSLs of Isaac and Cinch look, they are still pretty useless to me in their current state. Instead of reinventing the wheel, however, I will probably fork Isaac or Cinch and create a “still lightweight” framework based on that. In the end, I am planning to have an API which effectively allows code like the following:

on :channel, /^print my host$/ do
  # in this example, we will work with an instance of User,
  # instead of a plain string with just a nickname. Same goes for `channel`
  channel.send "Your host is #{user.host}"
end

on :channel, /!annoy (.+)$/ do |target|
  # indeed a helper for cases in which we have to deal with strings
  Channel(target).each_user do |victim|
    victim.send "hi, you have been annoyed!"
  end
end

Ruby benchmarks

2009-11-26T22:00:00Z

The following is a list of benchmarks done with the Ruby programming language, mostly showing that different ways of doing the same thing actually can have quite an impact on the overall performance of your code. And because this list is by no means “complete” or “rich” yet, you are asked to post your own benchmarks, which will then be added to this article.

These benchmarks were all made on a same system with the following setup:

System: Gentoo, 2.6.29, Pentium 4, 3.20GHZ
Ruby: ruby 1.9.1p243 (2009-07-16 revision 24175) [i686-linux]

Last update: 2009/11/27

Subject	Link	Conclusion
Check if a string contains a certain character	gist	include? is the most natural and fastet way.
Does slicing an array really cost us much?	gist	No.
Single-quoted vs. double-quoted strings	gist	When no interpolation is used, both kind of strings are equally fast.
Split: Regexp vs. String	gist	Strings are faster.
Replacing a single character: gsub vs. tr	gist	tr is way faster.
Deleting a single character: gsub vs. tr vs. delete	gist	No difference between delete and tr, gsub is again slower.
Deleting multiple characters: gsub vs. tr vs. delete	gist	A chain of deletes isn’t only ugly, but slow as well. tr wins again over gsub, isn’t any faster than delete though.
gsub: Regexp vs String	Derived from the tests with replacing a character	gsub is faster when used with regexps.
Construcing regexps	gist	When possible use literal regexps.
String interpolation in regexps	gist	Regexp.new seems to be slightly faster, if you need it to be interpolated every run.
Matching regexps	gist	=~ is faster than match, because it does not have to create a MatchData object.
Structs: Inheritance vs. assignment	gist	No noticeable difference.
A comparison of YAML and Marshal	Link

Dynamically defining a method on a single instance in Ruby

2009-10-10T22:00:00Z

Given the following initial code…

class Dog
  def initialize(name)
    @name = name
  end

  def bark
    puts "woof"
  end
end

charlie = Dog.new("Charlie")
lucas   = Dog.new("Lucas")

… one might want to teach one of the dogs a new trick, while the others shouldn’t automatically learn it. One possible solution could look like the following:

def charlie.jump
  puts "wee, #@name is jumping around"
end

Another way would be to provide a teach_trick method, which dynamically defines a new method:

class Dog
  # [...]
  def teach_trick(name, &block)
    (class << self; self; end).class_eval do
      define_method name, &block
    end
  end
end

charlie = Dog.new("Charlie")
lucas   = Dog.new("Lucas")

charlie.teach_trick :dance do
  puts "#@name is dancing around!"
end

charlie.dance # >> Charlie is dancing around!
lucas.dance   # ~> -:23:in `<main>': undefined method `dance' for #<Dog:0x9a62e70 @name="Lucas"> (NoMethodError)

What this does is it defines a new method on the eigenclass of the instance of a dog, so when the called method, dance in this case, is not found in the instance methods of the class Dog, it searches in the eigenclass of the instance instead.

One advantage of this solution is that the actual implementation of how new tricks are taught is hidden, the user does not need to know he is defining a method anymore, he is just teaching his dog a new trick. Another example is when you have to define a bunch of methods depending on some input, like a hash, which is completly dynamic.

One disadvantage, on the other hand, is that calling methods defined by define_method is a bit slower than those defined by def, but the factor is so small that you probably won’t notice it in real applications:

Benchmark.bmbm(10) do |x|
  x.report("eigenclass:") { 1_000_000.times {d1.trick} }
  x.report("instance def:") { 1_000_000.times {d2.trick} }
end
# >> Rehearsal -------------------------------------------------
# >> eigenclass:     0.440000   0.000000   0.440000 (  0.453996)
# >> instance def:   0.320000   0.000000   0.320000 (  0.322817)
# >> ---------------------------------------- total: 0.760000sec
# >>
# >>                     user     system      total        real
# >> eigenclass:     0.450000   0.000000   0.450000 (  0.451315)
# >> instance def:   0.310000   0.000000   0.310000 (  0.308632)

About getting help on IRC

2009-10-04T22:00:00Z

There is one thing that either we, the helping people, or those that search for help with their code don’t get, and that’s what kind of answers are to be expected.

Every too often I see people joining certain channels, in particular #ruby and #ruby-lang, asking questions about why their code doesn’t work or how to implement feature XY. And of course they show others their existing code, which usually isn’t too great, looking like a direct port from another language or simply like the one of a newbie (not that being a newbie is bad in any way, we all started as one).

Usually what follows isn’t a straight forward answer like “write this to get your feature” or “change that on line X to get it working” but a general discussion about the whole piece of code, how one shouldn’t write it like that at all and that, if it was done right, the problem wouldn’t have occured at all.

The purpose of this definitely is not to offend, discriminate or degrade anyone, but to teach the people how to code properly, how they can save time, nerves and money, and what it takes to become a good programmer. Programming isn’t about just hacking together a few lines that, by chance, end up doing what you expected them to do, but to create a well designed application, that can be maintained and extended at need.

Nevertheless we can see two completly different kinds of reaction. The first one is the person being thankful for all the great tips by the “pros”, even though he might be confused because of all the information at first. The other one though gets angry and tells everyone how he “wanted to fix a problem, not rewrite the whole damn thing from scratch”, generally feeling offended by all the well-meant advices.

So, the question is, who is right? Are we supposed/allowed to comment on the overall quality of the publicly posted code, or do we only exist to provide copy&paste solutions to people desinterested in quality of design? In my opinion, most if not all of those that spend their time helping others, want to make a change, want to educate, as opposed to just providing solutions that work in the given context, not fixing the bigger, the real problems.

I think that if the majority of help seeking people wanted us to stay shut about the code in a whole and just expected us to provide help with what exactly they asked, we would end up with not many people helping anymore.

So, what can you do in case you need help with a problem but do not want any help with your design and patterns? Well, the easiest option clearly is not to join those channels and try to figure it out by yourself. But I think another, not too bad, option is to actually seek help in those channels, but by explicitly saying that you only want (need is a bad word, because you probably do need) help with a certain problem and that you either have no time for or no interest in other advices. While this might put off some people you eventually will end up with one or another helping you with your particular problem, not wasting his, her or your time with information other than the ones you need.

In my opinion it definitely is wrong to ignore those advices made by professionals, but it is even more wrong to get angry about them. They’re well-meant, usually correct and you should respect them if you want to program in a proper manner. Yet, if you do not care about any of that, at least be so fair and tell us in advance so we can save our time.

Update, as of 2009/10/07

Okay, I’ve read all the comments about my article (both the ones posted on my blog and the ones on reddit) and it looks like I made one major mistake when writing this article: I was only thinking of the two main Ruby channels on FreeNode. While my article, in my opinion, totally holds true for those two channels it might as well be true that certain other channels are literally polluted by people who actively try to bash and piss off newbies. Alternatively, other channels seem to have a high amount of people with no deeper knowledge, who, nevertheless, try to force their opinion upon others, maybe not even backing it up.

So, what are people with problems supposed to do then? Avoid IRC alltogether? Hope to be lucky to have picked a language with a decent community? Only try to get help on IRC if you are able to tell apart bollocks from qualified help? I don’t know, but maybe you got the ultimate solution? And if so, maybe you don’t mind sharing it with us? Comments appreciated!

Getting rid of Google Analytics

2009-08-07T22:00:02Z

Ever since I started using Google Analytics, I felt bad about using it. Yes, I belong to those kind of people that think that Google is collecting way too much information about us and especially Google Analytics is making it terribly easy to track us, seeing how many websites are using it.

So I finally got my ass up and decided to search for a self-hosted, open source alternative. And that wasn’t even too hard, after just a couple of minutes I came across Piwik, a PHP+MySQL solution, which didn’t even look too bad.

Piwik has almost all the features that Google Analytics has, plus a plugin architecture which allows you to easily extend it (now try doing that with Google Analytics). It has support for a load of search engines, one can add new ones, it has a “Live!” widget for live display of visitors (yeah, absolutely unneeded but still a nice gadget) and one can update the reports at any given time (whereas Google Analytics is updated every hour).

It also has a more detailed list of installed plugins, even though it lacks specific version numbers in the reports.

There are some drawbacks though:

No custom date ranges
“Unique visitors” only work by day, not week/month (even though this might be fixed already, but then the FAQ is outdated)
I am currently facing a bug in the report generation

But all those problems are nothing compared to the huge advantage of having all the data on your own server, not giving away any of it.

So for now I am using both Google Analytics and Piwik, and if it turns out to indeed run well, I will remove the Google Analytics code alltogether.

Apache behind a remote lighttpd proxy and the caveats

2009-08-07T22:00:01Z

Background

The subdomain I use belongs to a friend, who isn’t able to set A or CNAME entries, so he uses lighttpd + mod_proxy to redirect it to my server.

The Problem

And that’s where the problems started. Because of the proxy, the HTTP REMOTE_ADDR isn’t the one of the visitor anymore but the one of the proxy, which in turns collides with log analysis, GeoIP and alike.

The Solution

In general, a simple solution would be to use the Apache module “rpaf”, which uses the X-Forwarded-For header to properly set REMOTE_ADDR.

In my case, that wasn’t the whole solution though. Instead of an IPv4 address, the lighttpd installation of my friend transmitted an IPv6 one, which then again didn’t work with GeoIP either, as it only supports IPv4. In this case, the solution was to run a local lighttpd, which itself uses mod_extforward to make use of X-Forwarded-For, and then properly forward the request to Apache, this time using IPv4.

Oh, also important: rpaf has to be loaded before GeoIP, otherwise the latter uses the wrong IP for lookups.

Intention over implementation details

2009-08-07T22:00:00Z

Amos Wenger has a nice post about why hiding the implementation details (how something is done) isn’t wrong at all and how/why it is one of the main characteristics of high level languages.

Ready for a change?

2009-08-06T22:00:00Z

Yesterday, after facing more and more problems with my Debian installation on my VPS, I decided it was a good idea to try out something new, namely Gentoo on a server.

Well, to make a very long story short: After 6 hours of updating 60 packages and noticing how poor the I/O on the VPS is, I decided to give up and installed Debian again… Gentoo definitely shouldn’t be your choice when on an overcrowded VPS.

Oh, and since I am already writing a post, here is a script that is useful when setting up servers (and using public key auth):

#!/usr/bin/env sh
echo "Deploying public key on $1"
scp ~/.ssh/id_rsa.pub $1:
ssh $1 'mkdir ~/.ssh; chmod 700 ~/.ssh; cat ~/id_rsa.pub >> ~/.ssh/authorized_keys; chmod 600 ~/.ssh/authorized_keys; rm ~/id_rsa.pub'

In addition, as of 2009/10/07:

zenspider was so friendly to provide me with an even better way of deploying the public key, not requiring you to use scp which in turn means only one password prompt (it also does not need the use of chmod nor any temporary file):

#!/usr/bin/env sh
echo "Deploying public key on $1"
ssh $1 'umask 0077; mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys' < ~/.ssh/id_dsa.pub

Only call super when possible

2009-08-06T22:00:00Z

Today we were faced with an interesting question in #ruby-lang:

<Judofyr> is there a way to find out if it’s safe to call “super”?
<Judofyr> aka. it’s defined on the superclass

After some ugly solutions (like self.class.ancestors.any? and super rescue nil) apeiros came up with a nicer one:

<apeiros_> super if defined? super

Example:

class Test
  def foo
    1
  end
end

class Test2 < Test
  def foo; super if defined? super; end
  def bar; super if defined? super; end
end

t = Test2.new
t.foo # => 1
t.bar # => nil

"Smalltalk Best Practice Patterns" reviewed

2009-08-03T22:00:00Z

Today I read the first volume of Smalltalk Best Practice Patterns, or to be precise, a draft I found on Google. While I am a Ruby programmer and never even touched Smalltalk in my life, it is an undeniable fact that Ruby and Smalltalk have many things in common, or to put it in another way, that Ruby (or any real OO language really) was inspired by Smalltalk.

The book is, like the title already says, about the “best practice patterns”, which covers naming conventions, coupling, splitting up methods and much more.

The book starts with an introduction into what patterns are, what they are needed for and how those in the book were developed and describes the intention of the book by one simple sentence:

This book is about making code that works for you.

It then, basically, consists of a long list of different patterns, which are described by a title, patterns preceding and following it, the problem stated as a question, possible problems with the pattern, the solution and a discussion of the solution.

Two important things about the patterns in this book, that distinguish them from other collections of patterns, are that they are discussed (instead of simply saying that they are the way to go) and that they are put into context, as most patterns can’t work if not combined with others.

Having read the book, I can say, that most if not all of the patterns covered in the book are also applicable to the Ruby world, even though some of them had to be slightly adjusted due to differences between Ruby and Smalltalk. But those changes only affected details, like the names of methods, and not the idea behind them.

It was kind of hard for me to decide if and to whom I should recommend this book. But I think this puts it the best way: If you are already familiar with the language you are using and also interested in proper design patterns, you should absolutely read this book, provided that you are able to adapt some Smalltalk specific terminology to the ones that suit the language you use. And even if you don’t want to pick up the author’s patterns, the first 40 pages of the book still help you defining your own patterns.

Debian and the problem with precompiled packages

2009-08-02T22:00:01Z

On my desktop I use Gentoo. Not because I believe in the performance gain of compiling the packages, but because I like the freedom of choosing exactly what components I need, which is, thanks to USE flags, surprisingly easy in Gentoo.

The only problem: Compiling stuff takes its time, and especially on servers I don’t like having to wait minutes up to hours on installing a new package. (Not mentioning all the stuff about stable, running releases and so on, thats another topic). So I use Debian on my server, and most of the time, the precompiled packages weren’t too bad, either.

But today I faced a rather weird choice of the responsible people. It looks like they compiled the php-gd extension in a way, that doesn’t allow you to use the imageantialias() function.

After some short googling I found a blog post which explains how to get the sources (which, even in Debian, is possible), change the rules (configure/build flags) and compile it on your own.

But that doesn’t explain their choice to not include the function in the first place, not even providing an alternate package.

Dominik Honnef

Writing Go in Emacs

History of go-mode

Obtaining it

Installation

Features

Reading documentation

Formatting code

Managing imports

go-import-add

go-remove-unused-imports

go-goto-imports

Navigating code

Interacting with the Playground

On-the-fly syntax checking

Autocompletion

Snippets

Finding unchecked errors with go-errcheck

Conclusion

Go: On implicit type conversions, type identity and a little gotcha

Dealing with encodings in Go

It’s all UTF-8, right?

But isn’t Go aware of UTF-8?

And how do I deal with other encodings?

Conclusion

Breaking backwards compatibility in good conscience

Linux RAID + dm-crypt: Poor write performance

Electronics: My first experiences with various shops

bausteln.de

Futurlec

Reichelt

Pollin

Electronics: Starting small

Disregard Females, Acquire Hardware

What is Arduino?

What Arduino to get?

Starter kits

Buying in bulk

My shopping list

Electronics: Introducing myself to the world of electronics

Introduction

SketchUp Plugin: Selection Manager

Don't Abbreviate "Thanks," "Sorry" or "Please"

rsemantic – Latent Semantic Analysis for Ruby

Introduction

Latent Semantic Analysis

LSA in Ruby

The future of rsemantic

New API

Old API

Why You Should Try tmux Instead of screen

What screen is

An Alternative

The History of screen And tmux

Actual New Features

A Sensible Configuration Format

Scriptable

Emacs And Vi Keymaps

Mouse Support

Menus

Multiple buffers

BSD License

Differences to screen

Server Model

Window Splitting

Missing Features

Final words

Links

Newton and Cinch – One awesome framework

Are lightweight IRC frameworks too lightweight?

Ruby benchmarks

Dynamically defining a method on a single instance in Ruby

About getting help on IRC

Update, as of 2009/10/07

Getting rid of Google Analytics

Apache behind a remote lighttpd proxy and the caveats

Background

The Problem

The Solution

Intention over implementation details