Keep Your Eyes on Your Keyboard
I recently read Accidental Genius by Mark Levy. I did so begrudgingly; I usually don’t go for books that promise to unlock the power of my mind through special techniques. I’m glad that I could put aside my stubbornness because the book contained a wealth of advice on tapping into ideas that you may not even be aware you had.
Accidental Genius advocates a practice called freewriting. Freewriting is just timed bouts of word excess. Sit down with pen and paper, or your favorite full screen editor, and go to town. Don’t think, write. This seems counter-intuitive until you understand that writing is thinking. By turning your hands into your own personal dictation device you engage that part of the brain the controls motor neurons freeing your mind to wander into deeper recesses.
I’ve been doing this to some extent intuitively for years. I first started “journaling” a few years back based on the 750 words meme that rolled over the internet. It advocated writing 750 words a day. Didn’t matter what they were, observations, stories, thoughts, feelings, etc. Just get it all out on paper. I figured if it works for Hemingway it will work for me. Every night for months on end I did this as a means of “brain-dumping”; just get it all out on paper. You can come back later and edit it into something worthwhile if you want…or don’t.
Think of it as making bread. You knead your brain with these 750 words or freewriting sessions, forming new connections between synapses you didn’t even know were there. After you knead and rest your dough (mind) for a while, you can come back and form your thoughts into rolls suitable for baking and serving.
One tip of my own to add:
Keep your eyes on your keyboard
Mavis Beacon will of course disagree, but I’m not trying to teach you QWERTY typing. The act of writing is different than the act of editing. When you write do it with reckless abandon. Ignore punctuation, spelling and typing mistakes. When you look at your screen your internal editor is watching what you put on the page, backspacing, deleting, correcting and moving forward. Backtracking into an idea to correct something as minuscule as a dropped comma forces your brain into the parallel tasks of writing and editing at the same time.
Try This:
Write stream-of-conscious for 5 minutes while looking at your screen. Do what you normally would, correct your errors and keep going.
Then Try this:
Write for 5 minutes looking at your keyboard. You may have made a typo back there or forgotten to type a comma but who the hell cares, your typing at the speed of your brain and not your stupid fingers. Just write, you can come back and clean it up later.
Now Compare
Did you write more in the latter 5 minutes than the former? I always do and I always end up with more usable material than when I edit as I go. Separating the acts of writing and editing is the most important lesson I learned from Accidental Genius.
In the past six months I’ve spent less time dumping my brain on the page, but with the kick start from Accidental Genius I think I’ll start the practice up again with more regularity and include some of the tips and prompts from the book.
Coffee Roasting: Build or Buy?
For the last five years I’ve been roasting coffee in this:
My Frankenstein Turbo Crazy.
But alas, after five years of operating “outside of normal temperature ranges”, this happened:
It seems that convection ovens weren’t meant to have their temperature limiters removed and run for hours on end. The handle, which controls the reed switch to turn the heating unit off and on, melted right off the base.
I knew this would happen someday, and I had planned to build another machine with a leaf blower and a commercial heating element, with robot arms and a positron brain. Alas, the unhappy melting happened right in the middle of rebuilding a fence, writing a bunch of software and getting through several releases at the office. I was posed with the classic question all technology households are posed with: build or buy?
A guy has got to have his home-roasted morning brew. I made a rash decision. I ordered one of these:
A GeneCafe from Sweet Maria’s.
After a week of waiting the GeneCafe finally arrived so after fulfilling my daily obligations I came home and took it for a test run on some Rwanda NKanka Kinyaga. In Benjamin Franklin style, here is how five years of DIY roaster technology stacks up to the commercial grade:
| GeneCafe | Frankenstein | |
|---|---|---|
| Pros |
|
|
| Cons |
|
|
The verdict. After the initial two test runs I’ll give the win to the GeneCafe. If only for the sweet analoguish knobs. We’ll see how it holds up after half a decade.
What’s the plan for my beleaguered Frankenstein Turbo Crazy? Well, I’ve always wanted an outdoor water heater…
Update: The coasting issue I complained about previously is no longer a problem. Seems I just needed to read the instructions…
Seven Languages in Seven Weeks
Think about the way you think. Think about that thought, and this one. Did you think using words? Did you see the words? Sound them out mentally? If someone asked you describe yourself, you would probably think of a series of adjectives (at least if you’re an English speaker).
We think via language, spoken or written. It’s the source of our intelligence and in some ways the root of our consciousness. Helen Keller is quoted with communicating that:
When I learned the meaning of ‘I’ and ‘me’ and found that I was something, I began to think. Then consciousness first existed for me
The languages you learn are the languages you express yourself with. They mold the way that you think about things and create who you are within your own mind. I’ve written about this before and it’s not an entirely new concept.
Recently I’ve gone to great lengths to change the way I think. Finding new ways to solve problems, especially software problems, often involves learning new languages, syntaxes or paradigms. You can force Java or C to do just about anything, just as you can force the English language to describe just about anything, but it might be that by using Java instead of Haskell, you’re using the wrong tool for the job.
I wanted to expose myself to a breadth of different software paradigms in as little time as possible. Rather than reading dozens of tutorials, or poring through hundreds of pages of reference manuals to get maximum exposure, I bought a book I called Seven Languages in Seven Weeks. Packed into this dense little tome is an overview of seven syntaxes from different families and programming paradigms.
Ruby
The book begins with Ruby. It’s a fairly common syntax and I considered skipping this chapter. Indeed, with the relative ubiquity of the language I wondered why it had been included at all. In the spirit of playing along with the author I read through the sections and did the exercises as described. It turned out to be a good idea; some of the concepts around using method_missing as a DSL generator I had never put into practice.
From a comfort standpoint starting with a language you’re familiar with is also a bit like reading the introduction to a Latin grammar text book in English. I know the language and therefore the author can present his approach to me with words I can understand before I try to make my way through the rest of his presentation.
Speaking of presentation, Tate clearly has a grasp of basic pedagogy. From the beginning he uses a mneumonic device to help the reader put a face to the chapter and the methodology. For Tate, every language is like a character in a movie. They have their own personalities; something that makes them unique within the dozens of lexical environments out there. For Ruby it’s Mary Poppins. You know, syntactical sugar. Get it.
IO
After Ruby Tate introduces a language I had never heard of, Io. Just try searching for information about this little language on the web. You won’t exactly find the throbbing community that surrounds java or ruby to back you up. No, if you choose to use Io to solve something new, you’ll likely find yourself in uncharted terrority. Not necessarily a bad thing if your approach to the text is to learn new ways to think.
A prototype language, Io is described by Tate as Ferris Bueller. In use I got the distinct impression that Io was heavily influenced by Smalltalk; everything you send is a message, and their are nothing but senders and receivers of messages. Method or function? Not really, there are ‘slots’ with message handlers. Can they be construed as the same thing, abstractly yes, but that is to avoid thinking in a way that make the language unique. Sending messages between objects is a powerful concept, and will help you better understand Objective-C and Smalltalk.
Using Io feels a bit like working in JavaScript, the only other prototype language I have any experience with. The concurrency framework is dead simple and provides the reader with a taste of things to come from languages like Scala and Closure. In fact, the actor framework in Io is so simple and impressive it feels like a great environment in which to teach concepts of asynchronous behavior and concurrent development.
Prolog
After Io we get to Prolog, the most frustrating language paradigm for me to grasp in the book. Tate says Prolog is like Rainman. That must make me Tom Cruise.
The logic programming paradigm was at once the most fascinating and frustrating for me to study. At first I was enthralled. A language that I can plug values into and simply query against to get the answer like a super-powered database? Sign me up. I immediately found myself fighting the syntax. It took me some time to grasp the recursive nature of the language as well; no looping structures.
Solving the sudoku problem at the end is the best example of the power of Prolog and languages like it. Reducing a game to a couple of line of syntax, injecting the rules and simply asking questions is a beautiful way to solve many of the problems modern engineers are presented with …if Rainman doesnt drive you nuts along the way.
Scala
With Scala we take a detour back to familiar territory. Scala is the first variant on the java language I’d had the opportunity to use, so when I began the chapter I had some exposure to it. Most of the concepts in this language sunk right in.
Tate says we can think of Scala as Edward Scissorhands. He is the construction of spare parts and a lot of paradigms that already exist. I prefer to think of Scala as MacGuyver; It can do pretty much anything in a pinch. Scala was a comfortable environment to take a break in for a while. It sports functional programming paradigms like higher-order functions, while retaining many imperative concepts held over from C-based languages. Its also completely interoperable with Java, so all of those libraries we’ve grown attached to like joda and jsyn can be reused in the same lexical environment.
For concurrency Scala provides an actor system, much like Io. Tate clearly planned the book to address concurrency in a methodical way, first by introducing simple examples with Io, then advancing to Scala before diving headlong into the deep waters of Erlang and Clojure.
Erlang
Things get uncomfortable again as Tate introduces erlang. From the get go Erlang baffled me, and when it was revealed that it was modeled after Prolog, I understood why. The only language compared to an antagonist in a movie, Tate describes Erlang as Agent Smith from the Matrix. Tate says that this is due to the self-replicating capabilities of Agent Smith in relation to the fail-safes built into Erlang, allowing the user to build highly fault tolerant concurrent systems that “just won’t die”. I think it’s because Erlang is evil.
Erlang is clearly very powerful, so as with Prolog I struggled through the examples and problem sets. I still don’t feel like I fully grasp how to do anything useful with it. Of the languages in the book, I feel like this is the one I need to spend the most time with to really understand.
Clojure
Next we get a lisp. Clojure, a language fully compatible with the jvm is a lisp not at all unlike Scheme, minus a few parentheses. For Tate this language is like Yoda, no doubt due to the “reverse” notation of the arguments and the “inside-outness” of the code construction, at least compared to C.
Surprisingly, I took right to it. Of the new lexical environments this felt the most comfortable, but then, I’ve played with emacs a bit. The concurrency framework is not at all unlike scala with some notable additions. The concept of STM was awkward at first, but after fiddling with it for a while I was comfortable producing usable code.
The interoperability with Java is another major benefit to using Clojure. For Dijkstra’s Sleeping Barber problem, rather than struggle through writing a queue from scratch, I just borrowed the existing Java LinkedBlockingQueue, cranking up one actor to poll it, and another to deliver to it:
In just under 1000 parentheses the barbershop problem was solved. The wrapper around it is unnecessary, but then the whole solution is a little bit wordy for Clojure.
Haskell
Impressed as I was with Clojure, it was time to study the final language in the book, Haskell. I originally selected the book based on the inclusion of Haskell. For some time now I’ve wanted to take a crack at this pure functional, almost entirely mathematical language.
Compared with the ever logical Spock, I’m still dazzled by Haskell. Having read the chapter and gone through the exercises, I feel like I’ve only scratched the surface of what it can do. It’s unofficial tag is that it makes hard things easy, and easy things hard. That ain’t no lie. Try reading a file with it. Do something simple, like open a socket. I feels like pulling teeth. Now go write a Fibonacci solver. Chances are you’ll have cooked up something that can’t be done as succinctly or quickly as Haskell can do it. Of all the languages in the book, this is the one I intend to dig the deepest with.
Wrap-up
When learning anything, I generally feel like a breadth-first overview is the best method of getting started. When learning new ways to think, this breadth-first search seems even more important. Get all your options on the table, see what’s been discovered before deciding how to tackle the problem. Selecting a strategy to go deeper with is a decision that can always be deferred until you know what your strategies are (Of course, you can only defer for so long before you just need to make a damned decision based on what you already know).
The real value of Seven Languages is that it provides this kind of breadth-first overview. You may know Java or C already. That’s great. What else is out there? What can a language like Io make easy? Clojure will help you understand lisp. Haskell will help you understand any functional and improve your understanding of modern math. Scala will let you build damn near anything.
Tate’s progression makes a lot of sense as well. If I was creating a curriculum to prepare a developer for the real world, I would start a youngster out with something like Ruby. This is an obvious ramp into Java and C. Then I might introduce something like Io to explain prototype languages and concurrency in a simple way. This is an step towards a better understanding of both Javascript and Objective-C. Then I might start them on Scala. It’s maximum exposure to as many concepts as possible. From scala, Learning a functional language is made easier if the programmer has been using Scala’s higher-order stuff like fold and map, and is used to immutable variables. Tate’s text provides a decent way to do all of this, introducing a young developer not only to the syntaxes but to paradigms that are broad enough provide insight to damn near any language out there.
If you’re interested in seeing my solutions to the exercises and problem sets, you can find them here. I learned a lot along the way, and I think I achieved the goal I had set out to achieve: Learning new ways to think.
Be Careful With your Redis BitSets and Java
Fast, easy, realtime metrics using Redis Bitmaps
A while back a popular article hit Hacker News. Written by the guys over at Spool, it contained a slick methodology for storing metrics such as user logins per day, song plays by user, etc using using Redis BitSets.
How about a basic example. When a user logs in, set a bit in a bitset at the location of that user’s ID number. If you have a bitset allocated for each day, you can tell for any given day how many users logged in by looking at the cardinality of the bitset. Want to see if a particular user logged in on a particular day? Just check the location in the bitset for that user’s ID for the day in question for a 1 value. You can also perform more advanced logging, taking the union of multiple sets, or the disunion, to determine various statistics.
The theory behind it is simple and sound. It’s faster than hitting an RDBMS for values that are binary in nature, and the ability to apply basic set theory to your bitsets to analyze your metrics is quite powerful.
I began to use this method and the code examples on the Spool blog to create metrics in a variety of systems, not to mention create silly stuff like prime number tables. It only took a few implementations to realize that the code examples, taken at face value, don’t really work.
The Problem with BitSet.valueOf() and BitSet.toByteArray()
The heart of the problem lies in the output of Java’s default BitSet.valueOf() method. Here is one of the examples on the page:
import redis.clients.jedis.Jedis;
import java.util.BitSet;
...
Jedis redis = new Jedis("localhost");
...
public int uniqueCount(String action, String... dates) {
BitSet all = new BitSet();
for (String date : dates) {
String key = action + ":" + date;
BitSet users = BitSet.valueOf(redis.get(key.getBytes()));
all.or(users);
}
return all.cardinality();
}
If you use the Jedis setbit method to set all of your individual bits, then read the entire set out with BitSet.valueOf(), Java reads the bytes as though they were right to left, whereas Redis stores the values in a straight line. Left to right. The bit sex, as it is called, is reversed in this case, and you can’t possibly get an accurate bitset out of Redis if you retrieve it and convert it using plain ol’ BitSet.valueOf(). You have to have a ‘tweener method to flip the bit sex for you.
You might also think, though it isn’t in the examples, that simply performing a BitSet.toByteArray() would create a byte array appropriate for storage in Redis to be read back via redis.getbit(); Not so. Java uses its native byte order for each call. This confuses things greatly, because if you set the bitset using BitSet.toByteArray() and read it back using BitSet.valueOf(), everything looks correct. Try to read a bit out of this array and be prepared for a surprise.
Some helper methods to get you through…
I reported this to the guys over at Jedis and they are considering adding some helper methods in their Redis client to alleviate this.
Until then, you can use the helper methods that the guys at Jedis and myself created to get you through.
To get a BitSet out:
public static BitSet fromByteArrayReverse(final byte[] bytes) {
final BitSet bits = new BitSet();
for (int i = 0; i <; bytes.length * 8; i++) {
if ((bytes[i / 8] & (1 <;<; (7 - (i % 8)))) != 0) {
bits.set(i);
}
}
return bits;
}
To put a BitSet in:
public static byte[] toByteArrayReverse(final BitSet bits) {
final byte[] bytes = new byte[bits.length() / 8 + 1];
for (int i = 0; i <; bits.length(); i++) {
if (bits.get(i)) {
final int value = bytes[i / 8] | (1 <;<; (7 - (i % 8)));
bytes[i / 8] = (byte) value;
}
}
return bytes;
}
And you can see the gist I created to read bytes out and put bytes in, retaining the integrity both ways, and show how things do and don’t work:
You can run that locally to get something like a code story…
So, whether or not the guys over at Spool have an unfortunately named helper method that looks exactly like the native Java one, or use some other methodology to maintain bit order in their bitsets I can’t say. It goes without saying that you should check and double-check any ol’ method you pull off the streets.
A Big Day In Banos
We met Franco El Blanco in the lobby at 4:30 AM. After a few checks to make sure we had everything we needed Frank introduced us to our ride. It seemed that Cary and I couldn’t escape from Ecudar’s two primary modes of transportation: buses and single cab trucks. Cary rode up front, three across with Frank and our driver; I just hopped in the back.
Luckily, the baths were just a short ride away. Situated just north of town, Termas El Salado are a bit newer than most of the other local baths, owing to the fact that they were completely destroyed by a volcanic eruption in 1997. Rebuilt on it’s ashes, you can see the remains of the former site if you look closely at the surroundings.
“We can have a nice soak if we get there at opening time, before the the crowds.” Frank told us.
In America only a few events can illicit a crowd at 5AM. A new Harry Potter book. An iPhone release. In Ecuador, the daily opening of the baths during Carnival in Banos is equivalent. When we arrived the hot pool’s slurry of yellow sulphur and tan-skinned inhabitants was already beginning to look like human soup. The two warm pools were filled with children on flotation devices. Only the cold pool was left alone, save for the occasional contrast bather passing from take a quick dip to cool off.
We hung out for a while, moving from pool to pool to try them out. I did some contrast bathing myself. When I get back I’ll have to engineer a way to do this at home. An hour later when we left the sun was rising and so were the real bath-goers; we walked out passing a line of latecomers who cared very little about beating the crowd. After a quick breakfast at the Hostal we headed out for our next event: Canyoning.
Canyoning is simple. You hike to the edge of a cliff, attach some ropes to yourself and rappel down. Then you find another cliff and do it again. Think of it as the opposite of climbing (though you have to do some of that to get to the cliffs). In Banos, canyoning usually meant descending into the canyons created by the many waterfalls.
We met up with a group, donned our wet suits harnesses and performed our first descent. It was little more than a practice round to get us started. Our second drop consisted of being dropped off a sheer cliff into the mouth of a roaring cascade. It was fun, but so far the guides were doing all the work. I was beginning to wonder if they knew that Cary and I weren’t exactly amateurs at this. We took a long hike out of the canyon and met up as a group close to where we started.
“Ok, now we go to the next waterfall.” Announced Jonny, one of our guides.
At this point the rest of our group descended the hill off into the morning, while Cary and I climbed even higher with Jonny to another waterfall. This is where we got to do a couple of real drops into waterfalls with a little movement. We made a few more descents, then thoroughly soaked called it a morning on canyoning. We headed back into Banos to get cleaned up, grab a quick lunch and head off for our next little adventure: Cycling on the narrow, bus-filled roads of Ecuador.
Las Rutas de Cascadas, or the Route of the Waterfalls, is a cycling path from the city of Banos to nearby Puyo. You ride out of Banos on the main highway heading East and along the way pass eleven or so waterfalls.
Cary and I didn’t intend to make the entire ride to Puyo, which usually takes a day. We instead rode out assuming we’d stop about halfway. In what became another adventure in “Cary and Grant Don’t Know What The Hell They’re Doing”, we pedaled out of town with little more than a vague photocopy of a map that sort of showed you where to go. After riding a few miles without seeing any cascades, it became obvious that our map wasn’t to scale.
Yet we rode on into the waning afternoon sun, eventually stopping at our first waterfall, which was a roadside attraction equivalent to Rock City. There were food vendors everywhere. A zip line ran from one cliff launching tourists headfirst over the deep valley to the other side. There was also a tarabina, or cable car, for the less adventurous. We watched a few of the zipliners superman over the cut in the Andes before jumping on our bikes and heading on.
As we made the route, we passed several of these tarabina pitstops at each waterfall. I noticed several derelict and rusted out tarabinas, indicators of a tourist boom the area couldn’t sustain, perhaps even remnants of the country’s economic bust a decade prior. 
There are other attractions along the way. Much of the local power is generated from a small hydro plant nearby, and the ride across the bridge provides a view in the maw of the dam. There are several tunnels, though only one that cyclists are allowed to pedal through. After two dozen buses pass you by at an ear’s breadth, the sparsely populated detours around the tunnels are a welcome retreat from the stress of riding on main road. Small towns dot the route, where other tourist industries have found a foothold. In one town visitors can bungee jump from the bridge into one of the canyons. Cary and I rode past, tempted but not enough to risk a trip to the hospital in Ecuador.
Side Note – Healthcare in Ecuador: Frank informed us that for an Ecuadorian, healthcare is quite expensive. A trip to the local clinic costs $30, and in a country where a couple hundred a month is the living wage, that’s a big chunk. Often, Frank will loan the cash to one of his many employees in the hostal to take their children to the clinic. Often, the trip to the clinic takes a detour to the local shaman instead.
“There are buses running to take you and your bike back to Banos along the way until 8:00” the agent at the bicycle rental shop assured us. We hadn’t seen a single bus, truck or car carrying any bikes back to Banos and we’d ridden the better part of three hours. Darkness was descending and Cary and I were beginning to wonder how we were going to get back. We consulted our poorly scaled map at intervals to guage our progress. We had a decision to make. Ride on hoping we came across one of these mythical buses to take us back, or turn around and start riding back to avoid riding too late into the night on a narrow road in the Andes frequented by brake-averse bus drivers.
We rolled the dice and rode on.
An hour or so later we came across the little town of Rio Verde, which was too wrapped up in it’s own Carnival celebrations to pay any mind to two gringos on bicycles. We rode around for a while, certain that this had to be the place where we would meet one of these mythical buses. After making a few confused circles in what looked to be the most crowded area of town, we shrugged and rolled the dice again.
“A Banos?” A mustached man jumped up as we approached and pointed to his pickup truck. We must have nodded hard, he quickly opened the back and started to pull the bikes out from under us an throw them in the back. We sat down on the makeshift bench in the back and breathed a little easier knowing we wouldn’t be out on an unknown road in the middle of the Andes in the dark again.
Back in Banos we returned our bikes and grabbed dinner at a little parillada we’d been eyeballing called Le Chiminea. It was a fine end to our time in Banos, which in only two days managed to feel like a week. Our trip to the Helen of Ecaudor complete, we fell into bed, ready for our last bus ride, which would take us out of the Andes and towards the sea.
