Monday, August 25th, 2008

[Ed: I’m changing things up today and writing about Windows. Obviously, the day job is involved.]

I’ve been consistently running into a problem with signtool.exe refusing to work at random points throughout my day. It would complain that it requires capicom.dll v. or higher. Despite the fact that I had several capicom.dll files on my system (no, I don’t know why) and they’re all v, signtool.exe would refuse to use them.

I believe (and here I really want to stress that I’m not speaking from a position of authority!) that the problem stemmed from the dll registration system. When I tried to register capicom.dll (any of them) manually, regsvr32 would fail and report “invalid access to memory location”.

Some poking around on Google revealed the cause: Windows XP SP2 added support for the modern processor’s No Execute Bit (which prevents the execution of code in data segments of memory and provides some much needed security from buffer overrun attacks). Apparantly, Microsoft considers regsvr32 an attack vector or something and it won’t work correctly with this extra protection turned on.

So the answer was to disable it. This Microsoft Knowledgebase article describes the functionality (called Data Execution Prevention, or DEP, in Microspeak) and how to control it. I went with changing my boot.ini’s noexecute parameter to AlwaysOff. And signtool.exe magically works!

This is obviously not a “correct” solution as my system is suddenly more vulnerable to machine-pwning attacks. But it works and it lets me get back to doing my job. So I’m not going to spend any more time on it, despite the fact that I’m somewhat unhappy about how it all turned out.

Also: “capicom.dll” is possibly the most ridiculous file name I’ve ever seen. Every time I say it out loud, I want to punch myself in the face.

[Added 2008-08-27]: So, the symptoms are happening again. My fix didn’t actually fix it. It’s such an intermittent problem, that it’s hard to diagnose. I hate intermittent problems. Anyway. Bear this in mind before you go an disable the DEP support on your system.

regular, tech Comments Closed
Friday, August 15th, 2008

A Cocoa-devmailing list thread popped up recently wondering how to make SOAP calls on an Apple platform. If you follow it through to the conclusion, you’ll learn that Patrick decided to roll his own: he’s got a base class which communicates with the server and triggers the actual serialization via simple string patterns or simple xml parsing.

This is almost exactly what I’m doing. The biggest difference is that he’s using NSXMLParser and I’m using TouchXML.

My question: how many times are developers going to have to write the same code over and over again in isolation before Apple adds a decent SOAP API to all of their platforms? Web services aren’t going away. Microsoft isn’t going away. And as long as developers are using Microsoft’s stack to write web services, we’re going to have to consume SOAP. Microsoft’s tools just make it far too trivial to do SOAP for it to fade into obscurity any time soon.

regular, tech Comments Closed
Monday, August 11th, 2008


Apple’s CFNetwork documentation leaves a lot to be desired. At one point while figuring out how to use it, I believe I went on a rant where I described its contents as “vicious lies”. Now that I’ve actually figured it out, I’ve revised my opinion. The documentation doesn’t lie, exactly. It omits. And what’s worse is that I couldn’t find anything on the Internet which told the entire story. So, I’m using this blog post to try to fill that gap. With a little bit of luck, Google will find me. And with a lot of luck I’m not posting incredibly inaccurate information here. So. Caveat: I’m still learning OS X programming. Don’t take this as gospel; take it as a good place to start. Also, I’ve removed all error checking for brevity. You’ll need to add it back. Good luck. (We’re all counting on you.)

I’m going to talk about sending a very basic HTTP post request to a server which authenticates with NTLM and then reading the response data. My example makes the request synchronously (because my application currently makes the request synchronously). You could modify this to use threads, or you could use polling or run loops. I’m not doing any of these, but I believe the ideas are basically the same. Refer to the documentation for more details.

The basic strategy is:

  1. Build a CFHTTPMessageRef containing the HTTP message to POST
  2. Send it to the server
  3. Read the response back from the server
  4. See if authentication is required
  5. Create a new CFHTTPMessageRef that looks exactly like the one created in step 1
  6. Add authentication credentials to the new CFHTTPMessageRef
  7. Send it to the server
  8. Read the response back from the server
  9. Repeat steps 4 through 9, as necessary
  10. Retrieve the HTTP body from the server’s response and hand it off to the caller for further processing.

In higher level frameworks (like Microsoft’s .Net or even Apple’s NSURLConnection), a lot of these steps are done for you. Indeed, if you don’t need a feature of the CFNetwork stack, I highly recommend using NSURLConnection. Unfortunately, NSURLConnection cannot do NTLM authentication. So if you’re talking to a Windows web-server that’s expecting domain-level credentials, you’re stuck. It’s not so bad, though.

Build a CFHTTPMessageRef

This is easy enough that I’m just going to point you at the code:

   NSURL *myURL = [NSURL URLWithString:@""];
   NSData *dataToPost = [[NSString stringWithString:@"POST Data It Doesn't Matter What It Is"] dataUsingEncoding:NSUTF8StringEncoding];

   //Create with the default allocator (NULL), a post request,
   //the URL, and pick either
   //kCFHTTPVersion1_0 or kCFHTTPVersion1_1
   CFHTTPMessageRef request = CFHTTPMessageCreateRequest(NULL, CSTR("POST"), (CFURLRef)myURL, kCFHTTPVersion1_1);

   CFHTTPMessageSetBody(request, (CFDataRef)dataToPost);

   //Unfortunately, this isn't smart enough to set reasonable headers for you
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("HOST"), (CFStringRef)[myURL host]);
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("Content-Length"), (CFStringRef)[NSString stringWithFormat:"%d", [dataToPost length]);
   CFHTTPMessageSetHeaderFieldValue(request, CFSTR("Content-Type"), CFSTR("charset=utf-8"));

   return [NSMakeCollectable(request) autorelease];

Send it to the server and read back the response

This one might require a little more explanation. We’re going to

  1. Create a CFReadStream for the request
  2. Open the stream
  3. While bytes are available on the stream, read them
  4. Once there are no more bytes available, copy the kCFStreamPropertyHTTPResponseHeader property from the stream (This is the part I can’t find in the documentation: reading bytes from the stream won’t actually give you everything. The read stream keeps the HTTP response header bytes for itself and makes it available as a property. This is entirely unintuitive, especially since the documentation insinuates that you can append the bytes from the read stream onto an empty CFHTTPMessageRef to get the response. You can’t do this, because reading the bytes from the stream doesn’t give you everything you need. :( Also, you have to copy the property from the stream after you’ve read all the bytes. Otherwise, it might not be there. Again, none of this is in the documentation (or, if it is, it’s hidden. I couldn’t find it after a few weeks of searching.))
  5. Use the bytes read from the stream AND the information from the kCFStreamPropertyHTTPResponseHeader to actually construct a CFHTTPMessageRef containing the response

   CFReadStreamRef requestStream = CFReadStreamCreateForHTTPRequest(NULL, request);

   NSMutableData *responseBytes = [NSMutableData data];

   CFIndex numBytesRead = 0 ;
      UInt8 buf[1024];
      numBytesRead = CFReadStreamRead(requestStream, buf, sizeof(buf));

      if(numBytesRead > 0)
         [responseBytes appendBytes:buf length:numBytesRead];

   } while(numBytesRead > 0);

   CFHTTPMessageRef response = (CFHTTPMessageRef)CFReadStreamCopyProperty(requestStream, kCFStreamPropertyHTTPResponseHeader);
   CFHTTPMessageSetBody(response, (CFDataRef)responseBytes);


   return [NSMakeCollectable(response) autorelease];

Adding Authentication to an HTTP Request

I’m going to skip to step 6 for a moment, just so I can use this method in the next step (which combines steps 5 – 9 in a single loop). This is actually pretty simple (the only wrinkle is that you HAVE to use CFNetwork code and can’t use NSURLConnection) so I’ll skip to the code.

-(void)addAuthenticationToRequest:(CFHTTPMessageRef)request withResponse:(CFHTTPMessageRef)response
   CFHTTPAuthenticationRef authentication = CFHTTPAuthenticationCreateFromResponse(NULL, response);
   [NSMakeCollectable(authentication) autorelease];

   CFStreamError err;
   Boolean success = CFHTTPMessageApplyCredentials(request, authentication, CFSTR("username"), CFSTR("password"), &err);

Putting It All Together

Now, we’re going to actually make the HTTP request in a loop. The loop will let us see if we need to authenticate. (Since good HTTP authentication uses a challenge-response mechanism, you have to make multiple requests. It’s a shame the libraries don’t wrap this up for us…) Once we have a response back, we’ll get the body and use NSLog to print it to the console.

   CFHTTPMessageRef request = [self buildMessage];
   CFHTTPMessageRef response = [self performHTTPRequest: request];

   UInt32 statusCode;
   statusCode = CFHTTPMessageGetResponseStatusCode(response);

   //An HTTP status code of 401 or 407 indicates that authentication is    //required I use an auth count to make sure we don't get stuck in an    //infinite loop if our credentials are bad. Sometimes, making the    //request more than once lets it go through.
   //I admit I don't know why.

   int authCount = 0;
   while((statusCode == 401 || statusCode == 407) && authCount < 3)    {       request = [self buildMessage];       [self addAuthenticationToRequest:request withResponse:response];
      response = [self performHTTPRequest: request];
      statusCode = CFHTTPMessageGetResponseStatusCode;

    NSData *responseBodyData = [(NSData*)CFHTTPMessageCopyBody(response) autorelease];
   NSString *responseBody = [[[NSString alloc] initWithData:responseBodyData encoding:NSUTF8StringEncoding] autorelease];


In Conclusion

I hope that was both clear and helpful. I tried to keep my commentary to a minimum because I’m a better coder than I am a writer (though, I’m not a particularly good coder when it comes to ObjC and Cocoa/Carbon). I wrote this inside a WordPress edit window so I could keep formatting mistakes to a minimum; but that means I haven’t actually compiled it. It may not work as presented; but it should be enough to get you the jist of what to do.


regular, tech Comments Closed
Friday, August 8th, 2008

Over on Apple’s Cocoa-dev mailing list, Peter Lewis asked if there was a way to release a CGImageRef. It turns out that this is trivial. In a program which isn’t garbage collected, you can just cast any CFTypeRef to an id and autorelease it: [(id)myCarbonObject autorelease].

If your app might be run under the garbage collector, though, that’s not good enough. You need to run NSMakeCollectable on the CFTypeRef: [NSMakeCollectable(myCarbonObject) autorelease].

Fortunately, even though NSMakeCollectable is new in OS X 10.5 Leopard, it’s an inline function and is therefore usable in apps targeting 10.4 Tiger.

I was excited to learn this because I had to jump through some hoops to release Carbon objects in the CFNetwork code I wrote recently. I can now happily just autorelease it.

regular, tech Comments Closed
Tuesday, August 5th, 2008

After several weeks of playing with it, pouring over Apple’s somewhat cryptic documentation, and trying every combination of every flag and order or operation I could think of…I’m finally making SOAP requests with CFHTTPMessage instead of WSMethodInvocation.

I can now make the exact same requests (Ping, SearchListLoad, DataFormLoad) that I could before I started all of this…but I’m not using any Mac-only APIs. (It’s still OS X only, of course.)

So I consider that a victory. At some point in the near future, I will write a blog post about just how to make an HTTP post using CFHTTPMessage. I need to get this information into Google so no one else has to spend weeks figuring it out on their own.

But I’m just happy to have it working. I can now move on to the much easier and only slightly less-interesting task of implementing the rest of the web service methods I need.

regular, tech Comments Closed