~james/developer/blog » Uncategorized

Pragmatic PHP

James Williams — Mon, 08 Dec 2008 23:13:16 +0000

I wrote some PHP today. I didn’t mean to.

All I wanted was a way to integrate strategically tagged pictures from flickr into a website. I couldn’t find any ready-made solutions to do what I wanted (making sure the page blends in with the rest of the site is the number one requirement (aside from actually pulling in the pictures)), so I had to write it myself. And while my host is certainly capable of running ruby or python…I couldn’t figure out how to do it without running rails or something. With PHP, on the other hand, it’s as easy as creating a file with a .php extension. So PHP was definitely the path of least resistance.

I haven’t really done much with PHP. I used it in college once, but that project involved staying up for forty-eight hours while banging out bad code in the computer lab. I can’t really remember much from that time other than prefixing variables with dollar signs. Which is probably for the best. (Despite handing the project in a couple of weeks late, I got an A by hanging out with the professor for a few hours. It was that kind of class.)

So I banged out a web page to do what I wanted. And I realized just how hard it is to write good code in PHP. I’m sure it’s possible, but I certainly haven’t figured it out yet.

After just a couple of hours, I already had a mostly unmaintainable mess on my hands. There are global functions and it’s basically impossible to tell where my logic ends and my markup begins. I’ve already refactored things out once, but things have already deteriorated.

Until now, I’ve considered myself a fairly disciplined programmer. I’ve developed pretty good instincts for setting up class hierarchies, making sure that everything I write only has one responsibility, avoiding code smells, and things like that. Yet somehow, over the course of hacking out this little project, I’ve managed to churn out something more expected from a first year programming student. I’d like to think I did it a little faster than that hypothetical first-year would have, but still.

I think it’s because I don’t actually know PHP. In most languages, that would mean that I just couldn’t have done anything and would have had to learn the language first. But PHP has such a large (if somewhat disorganized) library and there are so many copy-and-paste examples on the Internet that not knowing the language wasn’t actually a barrier at all. PHP let me get it done anyway.

So, score one for immediate pragmatism but minus a thousand for technical debt, you know?

I’m sure it’s possible to write good, maintainable software if you know the language well. But I suspect it’s hard. It’s so easy to write crappy PHP code (and I know because I just did it) that I can’t really imagine it being any easier to write good code. I imagine, for the PHP programmer, it’s a constant struggle to do things the easy way and just get it done versus spending a few extra minutes to do it the right way.

But I’m glad I don’t have to do it.

So, I salute all of the hard working, smart PHP programmers who are writing solid code against fantastic odds. Better you than me.

First Steps With Ruby

James Williams — Tue, 02 Dec 2008 21:32:32 +0000

After watching two fantastic PDC videos, Deep Dive: Dynamic Languages in Microsoft .NET and IronRuby: The Right Language for the Right Job, I started doing some serious thinking about dynamic languages and .NET. At work (where we use VB.Net), we tend to throw around several different data types that are essentially duck-typed: objects with key/value collections that if it has a certain set of keys, it’s of type A and if it has another set of keys, it’s of type B.

What would happen if we combined these objects with the capabilities of a dynamic language?

I have no idea. With the exception of Objective-C, I don’t have much experience with dynamic languages. And Objective-C, sitting on top of C like it does, is hardly a candidate for running on the DLR. And it’s hardly as expressive as something like Python or Ruby, which are conveniently the first languages available on the DLR.

So before I could start experimenting with our stuff and dynamic languages, I had to actually learn a dynamic language. I picked ruby, mostly by coin flip. I’m not starting with IronRuby right away. I find I’m always happier when I can manage complexity and it seemed that adding a not-quite-finished runtime to the mix could cause me trouble that I wouldn’t be prepared to fix. So I installed the “official” Ruby package for Windows and got to work.

I started out by implementing Conway’s Game of Life in Ruby. Over the course of an afternoon, I was able to whittle it down to 91 lines by swapping out the procedural statements I’d written for more Ruby-ish idioms. The Game of Life has such a simple set of requirements (there are just four basic rules) that I didn’t have to think about what the program had to actually do and could instead concentrate on the language I was writing it in. I think this is going to be my go-to first project when learning a new language from now on.

After that, I started implementing the first few Project Euler problems (the first one is doable with just one line of ruby). It didn’t take long for the math to get hard enough that I was no longer concentrating on the language, though. So I stopped pretty soon into the problem set.

I’m not really sure what I’m going to do next. Possibly something like Tetris, but it’s all up in the air. But even with this limited exposure to the language, I’m starting to look at the code I’m writing day-to-day and saying “How would this be easier in Ruby?”. I think that’s pretty valuable and so far, I’m quite pleased with the language and want to get to know it better.

signtool.exe and capicom.dll

James Williams — Mon, 25 Aug 2008 15:10:32 +0000

[Ed: I'm changing things up today and writing about Windows. Obviously, the day job is involved.]

I’ve been consistently running into a problem with signtool.exe refusing to work at random points throughout my day. It would complain that it requires capicom.dll v. 2.1.0.1 or higher. Despite the fact that I had several capicom.dll files on my system (no, I don’t know why) and they’re all v 2.1.0.1, signtool.exe would refuse to use them.

I believe (and here I really want to stress that I’m not speaking from a position of authority!) that the problem stemmed from the dll registration system. When I tried to register capicom.dll (any of them) manually, regsvr32 would fail and report “invalid access to memory location”.

Some poking around on Google revealed the cause: Windows XP SP2 added support for the modern processor’s No Execute Bit (which prevents the execution of code in data segments of memory and provides some much needed security from buffer overrun attacks). Apparantly, Microsoft considers regsvr32 an attack vector or something and it won’t work correctly with this extra protection turned on.

So the answer was to disable it. This Microsoft Knowledgebase article describes the functionality (called Data Execution Prevention, or DEP, in Microspeak) and how to control it. I went with changing my boot.ini’s noexecute parameter to AlwaysOff. And signtool.exe magically works!

This is obviously not a “correct” solution as my system is suddenly more vulnerable to machine-pwning attacks. But it works and it lets me get back to doing my job. So I’m not going to spend any more time on it, despite the fact that I’m somewhat unhappy about how it all turned out.

Also: “capicom.dll” is possibly the most ridiculous file name I’ve ever seen. Every time I say it out loud, I want to punch myself in the face.

[Added 2008-08-27]: So, the symptoms are happening again. My fix didn’t actually fix it. It’s such an intermittent problem, that it’s hard to diagnose. I hate intermittent problems. Anyway. Bear this in mind before you go an disable the DEP support on your system.

SOAP on Apple Platforms: Still Painful

James Williams — Fri, 15 Aug 2008 13:40:25 +0000

A Cocoa-devmailing list thread popped up recently wondering how to make SOAP calls on an Apple platform. If you follow it through to the conclusion, you’ll learn that Patrick decided to roll his own: he’s got a base class which communicates with the server and triggers the actual serialization via simple string patterns or simple xml parsing.

This is almost exactly what I’m doing. The biggest difference is that he’s using NSXMLParser and I’m using TouchXML.

My question: how many times are developers going to have to write the same code over and over again in isolation before Apple adds a decent SOAP API to all of their platforms? Web services aren’t going away. Microsoft isn’t going away. And as long as developers are using Microsoft’s stack to write web services, we’re going to have to consume SOAP. Microsoft’s tools just make it far too trivial to do SOAP for it to fade into obscurity any time soon.

HTTP POST via the CFNetwork stack

James Williams — Mon, 11 Aug 2008 23:36:22 +0000

Introduction

Apple’s CFNetwork documentation leaves a lot to be desired. At one point while figuring out how to use it, I believe I went on a rant where I described its contents as “vicious lies”. Now that I’ve actually figured it out, I’ve revised my opinion. The documentation doesn’t lie, exactly. It omits. And what’s worse is that I couldn’t find anything on the Internet which told the entire story. So, I’m using this blog post to try to fill that gap. With a little bit of luck, Google will find me. And with a lot of luck I’m not posting incredibly inaccurate information here. So. Caveat: I’m still learning OS X programming. Don’t take this as gospel; take it as a good place to start. Also, I’ve removed all error checking for brevity. You’ll need to add it back. Good luck. (We’re all counting on you.)

I’m going to talk about sending a very basic HTTP post request to a server which authenticates with NTLM and then reading the response data. My example makes the request synchronously (because my application currently makes the request synchronously). You could modify this to use threads, or you could use polling or run loops. I’m not doing any of these, but I believe the ideas are basically the same. Refer to the documentation for more details.

The basic strategy is:

Build a CFHTTPMessageRef containing the HTTP message to POST
Send it to the server
Read the response back from the server
See if authentication is required
Create a new CFHTTPMessageRef that looks exactly like the one created in step 1
Add authentication credentials to the new CFHTTPMessageRef
Send it to the server
Read the response back from the server
Repeat steps 4 through 9, as necessary
Retrieve the HTTP body from the server’s response and hand it off to the caller for further processing.

In higher level frameworks (like Microsoft’s .Net or even Apple’s NSURLConnection), a lot of these steps are done for you. Indeed, if you don’t need a feature of the CFNetwork stack, I highly recommend using NSURLConnection. Unfortunately, NSURLConnection cannot do NTLM authentication. So if you’re talking to a Windows web-server that’s expecting domain-level credentials, you’re stuck. It’s not so bad, though.

Build a CFHTTPMessageRef

This is easy enough that I’m just going to point you at the code:


-(CFHTTPMessageRef)buildMessage
{
   NSURL *myURL = [NSURL URLWithString:@"http://myurl.com"];
   NSData *dataToPost = [[NSString stringWithString:@"POST Data It Doesn't Matter What It Is"] dataUsingEncoding:NSUTF8StringEncoding];


   //Create with the default allocator (NULL), a post request, 
   //the URL, and pick either
   //kCFHTTPVersion1_0 or kCFHTTPVersion1_1
   CFHTTPMessageRef request = CFHTTPMessageCreateRequest(NULL, CSTR("POST"), (CFURLRef)myURL, kCFHTTPVersion1_1);


   CFHTTPMessageSetBody(request, (CFDataRef)dataToPost);


   //Unfortunately, this isn't smart enough to set reasonable headers for you
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("HOST"), (CFStringRef)[myURL host]);
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("Content-Length"), (CFStringRef)[NSString stringWithFormat:"%d", [dataToPost length]);
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("Content-Type"), CFSTR("charset=utf-8"));


   return [NSMakeCollectable(request) autorelease];
}

Send it to the server and read back the response

This one might require a little more explanation. We’re going to

Create a CFReadStream for the request
Open the stream
While bytes are available on the stream, read them
Once there are no more bytes available, copy the kCFStreamPropertyHTTPResponseHeader property from the stream (This is the part I can’t find in the documentation: reading bytes from the stream won’t actually give you everything. The read stream keeps the HTTP response header bytes for itself and makes it available as a property. This is entirely unintuitive, especially since the documentation insinuates that you can append the bytes from the read stream onto an empty CFHTTPMessageRef to get the response. You can’t do this, because reading the bytes from the stream doesn’t give you everything you need. :( Also, you have to copy the property from the stream after you’ve read all the bytes. Otherwise, it might not be there. Again, none of this is in the documentation (or, if it is, it’s hidden. I couldn’t find it after a few weeks of searching.))
Use the bytes read from the stream AND the information from the kCFStreamPropertyHTTPResponseHeader to actually construct a CFHTTPMessageRef containing the response


-(CFHTTPMessageRef)performHTTPRequest:(CFHTTPMessageRef)request
{
   CFReadStreamRef requestStream = CFReadStreamCreateForHTTPRequest(NULL, request);
   CFReadStreamOpen(requestStream);


   NSMutableData *responseBytes = [NSMutableData data];


   CFIndex numBytesRead = 0 ;
   do
   {
      UInt8 buf[1024];
      numBytesRead = CFReadStreamRead(requestStream, buf, sizeof(buf));


      if(numBytesRead > 0)
         [responseBytes appendBytes:buf length:numBytesRead];


   } while(numBytesRead > 0);


   CFHTTPMessageRef response = (CFHTTPMessageRef)CFReadStreamCopyProperty(requestStream, kCFStreamPropertyHTTPResponseHeader);
   CFHTTPMessageSetBody(response, (CFDataRef)responseBytes);


   CFReadStreamClose(requestStream);
CFRelease(requestStream);


   return [NSMakeCollectable(response) autorelease];
}

Adding Authentication to an HTTP Request

I’m going to skip to step 6 for a moment, just so I can use this method in the next step (which combines steps 5 – 9 in a single loop). This is actually pretty simple (the only wrinkle is that you HAVE to use CFNetwork code and can’t use NSURLConnection) so I’ll skip to the code.


-(void)addAuthenticationToRequest:(CFHTTPMessageRef)request withResponse:(CFHTTPMessageRef)response
{
   CFHTTPAuthenticationRef authentication = CFHTTPAuthenticationCreateFromResponse(NULL, response);
   [NSMakeCollectable(authentication) autorelease];


   CFStreamError err;
   Boolean success = CFHTTPMessageApplyCredentials(request, authentication, CFSTR("username"), CFSTR("password"), &err);
}

Putting It All Together

Now, we’re going to actually make the HTTP request in a loop. The loop will let us see if we need to authenticate. (Since good HTTP authentication uses a challenge-response mechanism, you have to make multiple requests. It’s a shame the libraries don’t wrap this up for us…) Once we have a response back, we’ll get the body and use NSLog to print it to the console.


-(void)magicHappens
{
   CFHTTPMessageRef request = [self buildMessage];
   CFHTTPMessageRef response = [self performHTTPRequest: request];


   UInt32 statusCode;
   statusCode = CFHTTPMessageGetResponseStatusCode(response);


   //An HTTP status code of 401 or 407 indicates that authentication is    //required I use an auth count to make sure we don't get stuck in an    //infinite loop if our credentials are bad. Sometimes, making the    //request more than once lets it go through. 
   //I admit I don't know why.


   int authCount = 0;
   while((statusCode == 401 || statusCode == 407) && authCount < 3)
   {
      request = [self buildMessage];
      [self addAuthenticationToRequest:request withResponse:response];


      response = [self performHTTPRequest: request];
      statusCode = CFHTTPMessageGetResponseStatusCode;
      authCount++;
   }


    NSData *responseBodyData = [(NSData*)CFHTTPMessageCopyBody(response) autorelease];
   NSString *responseBody = [[[NSString alloc] initWithData:responseBodyData encoding:NSUTF8StringEncoding] autorelease];


   NSLog(responseBody);
}

In Conclusion

I hope that was both clear and helpful. I tried to keep my commentary to a minimum because I’m a better coder than I am a writer (though, I’m not a particularly good coder when it comes to ObjC and Cocoa/Carbon). I wrote this inside a WordPress edit window so I could keep formatting mistakes to a minimum; but that means I haven’t actually compiled it. It may not work as presented; but it should be enough to get you the jist of what to do.

p>References:

Autoreleasing CFTypeRefs

James Williams — Fri, 08 Aug 2008 13:28:11 +0000

Over on Apple’s Cocoa-dev mailing list, Peter Lewis asked if there was a way to release a CGImageRef. It turns out that this is trivial. In a program which isn’t garbage collected, you can just cast any CFTypeRef to an id and autorelease it: [(id)myCarbonObject autorelease].

If your app might be run under the garbage collector, though, that’s not good enough. You need to run NSMakeCollectable on the CFTypeRef: [NSMakeCollectable(myCarbonObject) autorelease].

Fortunately, even though NSMakeCollectable is new in OS X 10.5 Leopard, it’s an inline function and is therefore usable in apps targeting 10.4 Tiger.

I was excited to learn this because I had to jump through some hoops to release Carbon objects in the CFNetwork code I wrote recently. I can now happily just autorelease it.

Quick Status Update

James Williams — Wed, 06 Aug 2008 01:18:27 +0000

After several weeks of playing with it, pouring over Apple’s somewhat cryptic documentation, and trying every combination of every flag and order or operation I could think of…I’m finally making SOAP requests with CFHTTPMessage instead of WSMethodInvocation.

I can now make the exact same requests (Ping, SearchListLoad, DataFormLoad) that I could before I started all of this…but I’m not using any Mac-only APIs. (It’s still OS X only, of course.)

So I consider that a victory. At some point in the near future, I will write a blog post about just how to make an HTTP post using CFHTTPMessage. I need to get this information into Google so no one else has to spend weeks figuring it out on their own.

But I’m just happy to have it working. I can now move on to the much easier and only slightly less-interesting task of implementing the rest of the web service methods I need.

Commenting Approach

James Williams — Wed, 30 Jul 2008 13:39:53 +0000

I’m unhappy with the FriendFeed comment plugin. I’m not knocking its author: I think it’s the best that can be done with the current FF API. But it just doesn’t work well as the only way to comment on a post.

So I’ve integrated Disqus commenting. I think I like it better than the standard WP commenting format. I’ll just have to see how it goes. Right now, it’s set up so registered Disqus users can comment immediately; unregistered folks will be tossed into a moderation queue.

I understand that this is mostly academic, of course. I don’t expect to drive much discussion. But it’s still amusing to try out.

Quick Note Regarding NSURLConnection and NTLM

James Williams — Wed, 30 Jul 2008 12:15:05 +0000

As near as I can tell, NSURLConnection does not support NTLM authentication.

So, if you are considering implementing something using NSURLConnection: take a moment to make sure you will never talk to a server expecting you to authenticate via NTLM (which is IIS as near as I can tell) before you waste two evenings doing it.

That being said, NSURLConnection is really cool and I wish I could use it.

For my next trick, figure out how to do this with CFHTTPMessage, which does provide kCFHTTPAuthenticationSchemeNTLM for authentication.

*sigh*

Accessing Instance Variables in ObjC

James Williams — Mon, 28 Jul 2008 16:07:17 +0000

A few months ago, I spent a few days searching around to see if I could figure out how to determine the instance variables on a class in ObjC. I knew it was possible because Key-Value-Coding will do it if it can’t find the well-named accessor methods for a keypath. But all of my searching proved fruitless and I went with a different method of accomplishing my end goal.

Well, it looks like the answer has appeared in my feed reader on an entry from Matt Gallagher’s Cocoa With Love blog:Key Value Information.

I’ll just quote him (from the very end of his post):

Similarly, if you were interested in key value coding that may access instance variables directly, you could use class _getClassVariable to get any instance variables with the same name as the key.

The leading underscore on the method name indicates that it’s undocumented and internal to Apple’s implementation. So using it isn’t necessarily the smartest thing in the world. But sometimes, you’ve gotta do what you gotta do.

Now that I have a method name, I’ll probably resume my search for information on this technique and will obviously report any findings here.

Update: July 29, 2008

I did some poking around in the Objective-C 2.0 Runtime Reference and found the class_copyIvarList function which returns an array of Ivar types. Ivar is a struct containing fields for the ivar_name, ivar_type, and offset. Combined with object_getIvar, I think I would have been much closer to achieving my original goal.

I don’t know why I couldn’t find this before. I guess I was just looking in the wrong places.