What's happening in Go tip (2014-01-10)

It’s 2014, we’ve survived Christmas and New Year’s Eve and the Go team has begun work on the next release. And thus the What’s happening in Go tip series has started again.

For this iteration of the series, there will be some small adjustments. The main change will be the lack of a strict weekly cycle. There might be multiple articles a week, or none. This is partly due to personal reasons and partly because it allows a more flexible approach to reporting changes. In the same vein, individual articles might be shorter than before, for the sake of addressing new developments faster.

The second change is the addition of coverage of proposals and discussions that haven’t seen any code changes yet. This is due to the fact that Go 1.3 will see some big changes (primarily the planned rewrite of the compilers in Go) that should be covered early on.

In this issue of the series we will, however, concentrate on the new sync.Pool type since it’s the first major addition to the standard library in Go 1.3.

Addition of the `sync.Pool` type

Relevant CLs: CL 41860043, CL 43990043, CL 37720047, CL 44080043, CL 44150043, CL 44060044 CL 44050044, CL 44680043, CL 46010043

Projects like the JVM invest a lot of time and effort in improving their garbage collectors to battle the high amount of garbage they need to deal with. Go, on the other hand, generally takes the approach of avoiding garbage in the first place, requiring a less sophisticated garbage collector by giving control back to the programmer.

To this end, several parts of the standard library have pools of reusable objects. The regexp package maintains state machines for concurrent use of a regexp, the fmt package has printers and other packages have their own pools as well, or would like to have them.

This approach, however, has two issues. The obvious one is code duplication: all packages have to implement their own pools, even though the implementation often is identical. A subtler issue is the lack of draining the pools. These simple implementations never release memory, which defies the point of a garbage collected language and can lead to unnecessarily high memory usage.

Because of this, Brad Fitzpatrick suggested adding a public Cache type to the sync package. This suggestion triggered a lot of discussion. Should Go provide such a type in the standard library or should it instead be private? Should it actually evict items, and if so, when? Should it even be named Cache, or rather Pool?

Let me explain the difference between a cache and a pool, and why it is important for this discussion. The type that Brad Fitzpatrick requested is actually a pool: A set of interchangeable values where it doesn’t matter which concrete value you get out, because they’re all identical. You wouldn’t even notice when, instead of getting a value from the pool, you get a newly created one. Caches, on the other hand, map keys to concrete values. A prominent example are disk caches, where files from a slow storage medium are cached in the system’s main memory for faster access. If the cache contains values for the keys (in this example file names) A and B, and you request the value associated with A, you definitely do not want to get B out. The fact that values in a cache aren’t identical also adds a lot more complexity to their eviction policies, namely the question of which values to evict when. The Wikipedia article on cache algorithms lists 13 different algorithms, from the well known LRU cache to more complex ones like the LIRS caching algorithm.

With that out of the way, the only real question concerning our pool is when to drain it. And pretty much every possibility has been suggested: somewhen before GC, somewhen after GC, clock-based or by employing weak pointers. And all suggestions had their own drawbacks.

After a lot of back and forth, Russ Cox eventually suggested an API and draining policy that was very simple: Drain the pools during garbage collection. This proposal reminds us of the fact that the purpose of the Pool type is to reuse memory between garbage collections. It should not circumvent garbage collection, it should only make it more efficient.

Brad implemented said proposal in CL 41860043 and, after some more back and forth, the CL was committed to the Go repository. Of course this CL wasn’t the end of the story. First, all existing pools had to be replaced with sync.Pool. This happened in the CLs CL 43990043, CL 37720047, CL 44080043, CL 44150043, CL 44060044 and not in CL 44050044. CL 44050044 was special in that it tried to use sync.Pool in the encoding/gob package, which has local free lists. And local is the keyword. The pools in encoding/gob belong to instances of a Decoder and are both short-lived and already handled appropriately by the GC. A free list would exist as long as the decoder did, and then be thrown away. Russ Cox commented on the CL, making clear what the purpose of sync.Pool is, and what it isn’t. To this end, Rob Pike submitted and committed CL 44680043, which expands the type’s documentation to state its purpose more clearly.

Pool’s intended use is for free lists maintained in global variables, typically accessed by multiple goroutines simultaneously. Using a Pool instead of a custom free list allows the runtime to reclaim entries from the pool when it makes sense to do so. An appropriate use of sync.Pool is to create a pool of temporary buffers shared between independent clients of a global resource. On the other hand, if a free list is maintained as part of an object used only by a single client and freed when the client completes, implementing that free list as a Pool is not appropriate.

From the comment (and prior discussions) it’s also apparent that the addition of sync.Pool is tentative and might be reverted before the release of Go 1.3, if it doesn’t live up to its promises. This is also tracked by Issue 6984.

While this marks the end of this exposition on sync.Pool, it’s not the end of the pool’s journey yet. There’s still CL 46010043, which improves the very simple initial implementation to become better suited for concurrent use, but it hasn’t been reviewed yet at this point.

Small change to the development process

Beginning with the Go 1.3 cycle, some small adjustments have been made to the development process. These changes only affect people who are either directly involved in the development process or who, like me, follow changes closely.

A new mailing list, golang-codereviews, has been created and made the default Cc target of new CLs, instead of the old golang-dev. The idea behind this is to reduce the noise on golang-dev, to allow actual conversations.

At the same time, a new dashboard has been created, allowing committers to easily track open issues and CLs. Anyone who is interested in how the Go team works is encouraged to read the usage instructions for the new dashboard.