Friday, April 25th, 2014

Python for Data Analysis
Python for Data Analysis by Wes McKinney
My rating: 5 of 5 stars

In my office, we spend a lot of time in the database. As such, we tend to become fairly adept at analyzing data with SQL: join some tables on interesting columns, group by other interesting columns, sprinkle in some aggregates, and pretty soon you have yourself a table of answers.

The relatively new windowing functions added in SQL Server 2012 let you do even fancier analysis (at the risk needing to understand some new syntax).

Yet, sometimes, a raw table of SQL results just isn’t enough. You might want to have access to some interesting statistical functions like the standard deviation or variance. You might want to correlate your data with other data available via a web service or just create a nice looking chart.

In these cases, SQL may not be the best choice. In recent years, Python has become an extremely popular language for doing data analysis. With libraries like pandas, numpy, and matplotlib, Python has rapidly become an extremely credible challenger to established special-purpose envrionments like R and SAS. Combined with these libraries, the dynamic ease-of-use of Python becomes perfect for the sort of data analysis tasks we find sometimes find ourselves trying to approach.

This book is an excellent introduction to the subject. It’s written by the creator and lead developer on the pandas project, which provides the table-like data structures that make this sort of thing so comfortable for us SQL developers.

By focusing on data analysis, McKinney assumes no real knowledge of Python programming and even provides a “Python Language Essentials” at the back of the book. Assuming that you’re already a competent programmer in Java, C#, VB.Net, or some other procedural language, you can easily pick up the basics of Python just by working through the examples in the book.

And that’s highly recommended because the examples are a lot of fun! It’s fun to grab stock data from Yahoo, run it through some computations, and then graph the result. McKinney strongly encourages the use of the IPyton shell and I echo that (especially the use of the notebook). It adds a strong sense of interaction to the standard Python REPL — indeed, it feels a lot like tweaking queries and hitting F5 in SQL Server Management Studio. IPython notebooks are now easily my favorite way to experiment with new programming methods.

Working through this book is a great way to spend a weekend. And when you’re done, you’ll be able to dazzle your designers and product owners when they ask for data about something. Instead of making them squint at a grid of SQL results, you can hand them charts and graphs that will make it look like you spent hours in Excel.

Sunday, April 20th, 2014

The Golem and the Jinni
The Golem and the Jinni by Helene Wecker
My rating: 3 of 5 stars

This book does not lend itself to an effusive review.

It’s fine, really. It has an interesting premise. The occasional-flashback story structure is useful for building to the climax. The characters are more-or-less likable.

It’s fine.

It could be so much more than it is, though. The prose is, at best, workmanlike: it gets the job done but is utterly forgettable, sentence after sentence.

The setting could could have propelled the story to greatness: a look at at pre-20th century New York’s immigrant communities and class system through the eyes of two total outsiders. But it’s not to be. There’s almost no difference between the Jewish characters and the Syrian characters except that one group drinks tea and schannps while the other drinks coffee and araq.

Similarly, the opportunity to have a meditation on the nature of free-will is squandered by just shrugging it away. The nature of western theology is winked at but unexplored.

The book even proposes multiple (possibly conflicting) magic systems without really delving into it.

So, maybe all that’s fine. It’s clearly intended to be a fairy-tale and fairy-tales are free to leave all of that sort of thing behind. But this is a fairy-tale without the wonder. And when you’ve stripped out the complexities and questions and wonders of life, history, and magic — you’re not left with much.

This book is largely competent and it’s a not-unenjoyable way to pass a few hours. But — with cookie-cutter characters, cardboard settings, and a disdain of complexity — it never really rises up to be anything else.

I wouldn’t try to stop anyone from reading it, but nor would I recommend it in our world that’s overflowing with fantasy books.

The Future of the Mind: The Scientific Quest to Understand, Enhance, and Empower the Mind
The Future of the Mind: The Scientific Quest to Understand, Enhance, and Empower the Mind by Michio Kaku
My rating: 1 of 5 stars

I can’t abide futurism. The best science fiction postulates an imaginary future society with imaginary future technologies and explores the present through a fantastical lens. Futurism, on the other hand, postulates that imaginary future because “why not?”.

Futurism is little more than making extravagant predictions while hand-waving away the very real technical issues that stand between the present and that predicted future. In my field of computer programming, we often tell stories of “the sufficiently advanced compiler”: a theoretically-possible program that be able to understand what we mean when we write computer programs and not just what we say and will use that understanding to rewrite programs to be faster and more correct than we could ever manage on our own.

It should be needless to say that this sufficiently advanced compiler does not really exist, even though the only thing standing in its way is sufficiently clever engineering. It turns out that sufficiently clever engineering is really hard.

Similarly, futurism pretends that all of their fantastical technical advances are just a matter of that same kind of sufficiently clever engineering. “This is theoretically possible, therefore we’re guaranteed to figure it out, and I don’t need to worry about the how because it’s just engineering.” is the song of the futurist — and once they’ve established that one or two fantastic technologies are inevitable they can pile advancement on top of advancement on it until you end up with future predictions that are barely distinguished from fairy tales.

It is, or (at least) should be, obvious that this book is a work of futurism. It has the word “future” in the title and everything. But, I’d hoped that Dr. Kaku’s experiences with actual physics would drive him to ground the work in the reasonable if not the possible.

Unfortunately, Dr. Kaku is extremely excitable. Excitability certainly has its place in science. I like my popular scientists to exude a sense of and wonder, but I’m also pleased when they can barely keep themselves from jumping up and down because science is just so cool. Unfortunately, Kaku quickly moves from excitement to breathlessness as moves without pause from wonder to wonder that neuroscience is making possible.

Well, might make possible.

Well, might show is theoretically possible.

One day.

It’s an engineering problem. Let’s assume it’s real and see what happens next.

And so on and so forth.

At one level, it’s exhausting. He never slows down to let you marvel at the mysteries of the brain or the Herculean efforts that researchers are making in order to unlock them. At another level, it’s extremely frustrating as he completely sacrifices the near-term in favor of looking centuries ahead. By focusing solely on the far-future potential (beaming consciousness around the solar system? Really?), he’s giving short-shrift to the work-a-day scientists who are relentlessly plugging away at the enigmas that are in front of them today.

But then again, I suppose: what should I expect from a theoretical physicist?

Dr. Kaku’s prowess as a theoretical physicist may also lead into the second most problematic part of this book (aside from my distaste for futurism in general): “I’m not an expert in this, but…”.

The most glaring example of this is when Kaku admits that he does’t know what he’s talking about but decides to try to define “consciousness” anyway. That’s the entire second chapter of the book, “Consciousness – A Physicist’s Viewpoint”. Instead of being embarrassed about trying to define something that the actual experts in the field have struggled with, he instead builds large portions of the book on top of this scaffolding.

Indeed, he seems quite proud of his definition. He gives it a name, “the space-time theory of consciousness” and refers to it by name again and again. I have my doubts about his theory of consciousness.

I don’t think it’s entirely wrong, but I also don’t think it’s entirely useful. I was also put off by the way he pokes fun at the homunculus argument (which more-or-less posits that there’s a “little person” in the brain driving our bodies) and then almost immediately names an imaginary “CEO” as the consciousness in his definition. I’ve read the entire book and I can’t really tell you the difference between Kaku’s CEO and the discredited homunculus.

If all you’re going to do is reduce the idea down to an ineffable “CEO”, what’s the point? And how can you build so much of your book on this topic?

Finally, Dr. Kaku’s insistence that so many wonderful things (“reverse-engineering the brain”, making full brain copies, beaming our consciousness to the stars on beams of light, controlling robots with our brain as if they were our bodies, etc.) are only a century out (two centuries out at most) seems perfectly analogous to the claims that useful fusion reactors are only fifty years away — claims that have been made continuously for over fifty years.

A scientist’s skepticism should require him to justify these claims with far more than he even attempts.

Ultimately, I found this book extremely unsatisfying. The interesting work being done today would make a fascinating book, but Kaku races past them to instead dive into limp science fiction which offers neither the technical rigor of the best “hard” sci-fi nor the reflection of our own society offered by “soft” sci-fi.

I can only recommend it as a reminder to not read non-fiction books with the word “future” in the title. They rarely go well.