October 5, 2006

Pimp My Code, Part 12: Frozen in Carbonite.

Carbon, briefly defined, is a compatibility library that ships with Mac OS X that enables older applications, written for Mac OS 9 and before, to run under Mac OS X with minimal changes (and a recompile). Carbon is descended from the original Mac Toolbox written in the early 1980s, and still shows signs of its Pascal and machine-language origins, even though it is now primarily accessed through and written in the C language.

Cocoa, briefly defined, is a new application environment that Apple got (and improved upon) when it bought NeXT in 1997. Cocoa uses a dynamic, object-oriented language (Objective-C) by design, and was the inspiration for modern languages like Java.

For a few years after the initial release of Mac OS X (10.0), there was speculation about whether Carbon or Cocoa would end up dominant. Many Mac programming groups inside and outside of Apple didn't know Cocoa and assumed it would be another flavor-of-the-month that would be quickly abandoned as unworkable, like Pink and Taligent and Copland.

Apple's official stance was initially that developers were encouraged to use either Carbon or Cocoa to write new applications; in the last year or so that message has quietly changed to just encouraging Cocoa, both because Cocoa is being enhanced faster than Carbon and because increasing numbers of Apple engineers are trying Cocoa and finding it an easier and faster to code environment within which to code.

But there are still people out there who use the Carbon APIs to program on the Mac, and there are still those who assume Carbon is "faster" and Cocoa is just "easier". While this has no basis in any timing tests I've seen, it's a myth that persists.

It should be noted that over the past eight years Apple has merged the Carbon and Cocoa runtime environments to the point where one can call Carbon routines from inside any Cocoa program, and vice-versa, so the idea that one environment is inherently "faster" is kind of crazy, although one can obviously fail to optimize one's code, and there is a partly-relevant question which goes: if you write the same number of lines of code in each language to do the same job, which would run faster?

The answer, in my experience, is Cocoa wins hands-down, because there's no such thing as a Carbon program that's the same number of lines as a Cocoa program that does the same thing. Carbon is wordy, in part because it is based on C, and in part because it still uses metaphors and programming conventions that were in vogue twenty to thirty years ago.

--

What's distressing to Cocoa programmers is that there are still critically important Apple APIs that are only available through the incredibly byzantine and ill-documented Carbon libraries, and some groups at Apple are still generating Carbon code, under the guise of "Core".

Now, let me be clear. I have nothing against "real" Core libraries, like CoreFoundation or CoreGraphics or CoreAnimation. These are all modern APIs that happen to be written in C, and they tend to have good to great support above them in Objective-C, either via separate APIs (like NSBezierPath in AppKit, which automatically emits the correct CoreGraphics calls) or via toll-free bridging (like NSStreams and CFStreams).

But some "Core" libraries aren't "Core" at all -- they are still using the conventions and structures of Carbon, and these libraries are nigh-impossible to use. QuickTime, Keychain / Security Services, Core Audio, Launch Services, Speech and Voice recognition -- these are the ones I've recently run up against. The problems with even well-written Carbon libraries are myriad:
  1. There is usually one, giant, all-encompassing "setAttributes" function and one "getAttributes" function for setting and getting every value associated with a Carbon "object", which requires to you laboriously build up and then laboriously unparse huge, special parameter structures for even the simplest call. For example, see one of the newer (v6.4) QuickTime calls, for setting parameters on a FireWire video camera: VDIIDCSetFeatures(VideoDigitizerComponent ci, QTAtomContainer *container). That one call is used to set a HUGE number of parameters, which is "simple," except, oh, it takes eighty lines of code to prepare for that one call, and the parameters you can set are not actually documented in the documentation. (This is not an exaggeration, I'll show Apple's own example code later.)

  2. Carbon calls with monolithic parameter blocks and anonymous parameters (like the one above) are not inherently self-documenting, and the documentation for them is often wrong and/or incomplete because the documenters' job is so much harder. For example, with the given documentation there's really no way you could intuit how to build up a legal parameter block to actually call VDIIDCSetFeatures() unless you also read the technote Apple wrote on it in 2005, which is very helpful, but was written a year too late for Delicious Library and several years after QuickTime 6.4 was released.

  3. Parameters passed into functions are passed by reference, whether they are going to be modified or not, which is just obscure. For example, see SecKeychainItemCopyAttributesAndData(), which takes the input "info" by reference, even though it's read-only!

  4. Carbon uses the same type, OSTypes (eg, 'sit!'), extensively for everything: return codes, building parameter blocks (specifying what you want to do, how you want to do it), loading QuickTime components, etc., which means reading a function description doesn't actually give you enough information to know what to actually pass it. (Eg: "Pass it some combination of four-character constants. Good luck!")

  5. Carbon memory management is a mish-mash of new and old metaphors, so you might easily find yourself using, in addition to CoreFoundation objects that can be retained and released, "Handles" or "Atoms", which are obtuse and easy to screw up.

  6. Carbon often uses FSRefs (although there are now often string-path equivalents) to refer to files, which are, again, not fun or easy to build or use.
Let me note that I'm not trying to slam Apple's Carbon programmers here; if I were a programmer on some Carbon toolkit I wouldn't necessarily try to break every metaphor everyone else was using, either -- I'd add my code and clean things up a bit where I could. (Or, actually, I would, but I would suggest we just rewrite everything in Cocoa, which is what I'm getting to in this post.)

Carbon has matured a lot in the last several years (eg, more Cocoa-like). But it's not enough. What I want to do, with this post, is encourage Apple to finally move those necessary but neglected frameworks all the way to Cocoa.

--

Ok, so I've been a bit abstract. Let's look at two function calls from two different Carbon libraries that are under active development and that you MUST use if you want their functionality -- there is not equivalent in any other framework. We'll take them apart, see why they are so hard to use, and then look at an ideal Cocoa API that would accomplish the same thing but be a zillion times easier to call and understand.

First off, let's look at how you might set the hue, saturation, sharpness, brightness, gain, iris, shutter, white balance, gamma, temperature, zoom, focus, pan, tilt, optical filter, focus point, and more on IIDC cameras.

Ok, wait, wait a minute. What did I just say? "IIDC cameras"? What the hell is an IIDC camera? And why do I care? Doesn't QuickTime isolate me from the low-level nonsense and just provide me with an opaque Sequence Grabber object which handles all the hardware bullshit and provides me with a single unified interface?

Well, the answer is, "Of cou--no. No, not really. Psych."

If you have a certain class of FireWire cameras which have a common way of setting their parameters, Apple has provided low-level calls to set those parameters that only work with those cameras, and deprecated the other function calls. Note that, if you have an older (or newer) camera, or a USB camera, like the internal USB iSight that ships with EVERY iMac and notebook Apple sells, these IIDC functions don't work for you, and you have to use the older functions (where available), even though Apple says they are deprecated. Wheee!

Ok, so assume you have one of these cameras, and you want to set, say, the gain on it. How would you do it? Well, normally, you'd spend months guessing at how to build up the parameter block to please VDIIDCSetFeatures(), but as of 2005, as I said, Apple has a technote, which shows how the gain can be set, in only eighty lines of code.

Seriously. Eighty. Let's look:

Setting the Gain on a Camera in QuickTime
ComponentResult ConfigureGain(SGChannel inChannel)
{
QTAtomContainer atomContainer;
QTAtom featureAtom;
VDIIDCFeatureSettings settings;
VideoDigitizerComponent vd;
ComponentDescription desc;
ComponentResult result = paramErr;

if (NULL == inChannel) goto bail;

// get the digitizer and make sure it's legit
vd = SGGetVideoDigitizerComponent(inChannel);
if (NULL == vd) goto bail;

GetComponentInfo((Component)vd, &desc, NULL, NULL, NULL);
if (vdSubtypeIIDC != desc.componentSubType) goto bail;

// *** now do the real work ***

// return the gain feature in an atom container
result = VDIIDCGetFeaturesForSpecifier(vd, vdIIDCFeatureGain, &atomContainer);
if (noErr == result) {

// find the feature atom
featureAtom = QTFindChildByIndex(atomContainer, kParentAtomIsContainer,
vdIIDCAtomTypeFeature, 1, NULL);
if (0 == featureAtom) { result = cannotFindAtomErr; goto bail; }

// find the gain settings from the feature atom and copy the data
// into our settings
result = QTCopyAtomDataToPtr(atomContainer,
QTFindChildByID(atomContainer, featureAtom,
vdIIDCAtomTypeFeatureSettings,
vdIIDCAtomIDFeatureSettings, NULL),
true, sizeof(settings), &settings, NULL);
if (noErr == result) {
/* When indicating capabilities, the flag being set indicates that the
feature can be put into the given state.
When indicating/setting state, the flag represents the current/desired
state. Note that certain combinations of flags are valid for capabilities
(i.e. vdIIDCFeatureFlagOn | vdIIDCFeatureFlagOff) but are mutually
exclusive for state.
*/
// is the setting supported?
if (settings.capabilities.flags & (vdIIDCFeatureFlagOn |
vdIIDCFeatureFlagManual |
vdIIDCFeatureFlagRawControl)) {
// set state flags
settings.state.flags = (vdIIDCFeatureFlagOn |
vdIIDCFeatureFlagManual |
vdIIDCFeatureFlagRawControl);

// set value - will either be 500 or the max value supported by
// the camera represented in a float between 0 and 1.0
settings.state.value = (1.0 / settings.capabilities.rawMaximum) *
((settings.capabilities.rawMaximum > 500) ? 500 :
settings.capabilities.rawMaximum);

// store the result back in the container
result = QTSetAtomData(atomContainer,
QTFindChildByID(atomContainer, featureAtom,
vdIIDCAtomTypeFeatureSettings,
vdIIDCAtomIDFeatureSettings, NULL),
sizeof(settings), &settings);
if (noErr == result) {
// set it on the device
result = VDIIDCSetFeatures(vd, atomContainer);
}
} else {
// can't do it!
result = featureUnsupported;
}
}
}

bail:
return result;
}

Wow, easy!

Remember, this is only for a certain class of cameras (including external iSights, but excluding internal iSights) -- if you want your code to work in all cases, you also have to call SGSetSettings() (or some such, I honestly don't know, as the focus calls I was trying to do don't have equivalents) to set up other kinds of cameras, except this call isn't as flexible, and, of course, nowhere in the documentation for the call does it say which settings you can actually set. Because, since everything in Carbon just takes and returns OSTypes (see sin #4), it's not self-documenting what kinds of parameters you can pass in to any method -- there are tens of THOUSANDS of OSTypes defined in the system, good luck finding the list of the ones your particular function takes and returns! They certainly are never listed in the documentation of the functions that take them.

Now, if Apple had used an enumerated type instead of an OSType, as such:
Imaginary QuickTime C Typdef
typedef enum {
kSequenceGrabberFocus, kSequenceGrabberZoom, kSequenceGrabberGamma,
[...]
} SequenceGrabberParameters;

Well, then we could just command-double-click on the function prototype in XCode, and we'd have our own documentation (after a fashion), in the form of the header file where the parameters are defined.

(On a side note, sadly, the AppKit and Foundation team just announced they are moving AWAY from using enumerated types in Cocoa methods and switching everything to take plain ints, because they HATE self-documenting code and hate Cocoa programmers. No, it's because they are worried that enumerated types could have unspecified sizes. But, seriously, Ali, this is totally broken -- enumerated types make reading and writing code MUCH easier, and they enforce sanity in switch() statements and if() comparisons. Please don't break this! [Update: actually, it's not quite as bad as it could be: the proposed new types aren't totally anonymous, they just are typedef'ed ints, but at least they are defined right below real enums, so command-double-click still works. Debugging and switch() and if () still aren't handled as nicely, though.]

Now, the above enumerated type would be helpful IF QuickTime had to stay with the giant, all-encompassing set-get functions. But must they? Well, no. For some reason some Carbon programmers think it's easier for them to add and remove APIs if they never commit to any particular functionality, so they use those giant anonymous set-get functions and then just have you build up a list of parameters yourself, with the idea being that it's a lot more flexible if they change the parameters out from under you than if they change the functions.

Acrazypersonsezwhat? Ok, I don't know what life was like under Mac OS 9, but nowadays we've got a well-defined system for deprecating APIs, and it works. And the advantages to not building up parameter blocks are huge. To wit, imagine the above gain-setting code, rewritten in a (hypothetical) Cocoa interface to QuickTime:

Setting the Gain on a Camera in Imaginary Cocoa QuickTime
ComponentResult ConfigureGain(QTSequenceGrabblerChannel *sequenceGrabblerChannel)
{
return [sequenceGrabblerChannel setGain:1.0];
}

Sure, four lines is less than eighty, but the observant amongst you will say, "Wait a minute, in the original code he checks to see if this is an IIDC camera first!" Well, yes, but, honestly, I should be able to call my mythical -setGain: on any physical sequence grabber, and if the user's particular hardware doesn't implement gain, QT should just ignore me.

More observant folks may exclaim, "But, he sets the camera's gain feature to be: on, manual, and raw control!" And I must calmly say, if I call -setGain: on a sequence grabber, it's implicit that I am putting it under manual control! Otherwise, what does setting the gain mean? I should not have to write this code.

Even more observant folks may add, "Well, but, he also has some crazy code where he either sets the value to the camera's raw maximum or 500..." to which I respond: I should be abstracted away from such nonsense. Parameter values should be normalized from 0.0 to 1.0 for me; where 1.0 is the hardware maximum, whatever it is. I should never have to query the hardware.

Now, it could be I'm oversimplifying a bit. I can imagine there are complexities in this code that require, say, TWO lines of Cocoa inside my imaginary rewrite of ConfigureGain(), not just one. But you can see where I'm going with this.

The fact that Apple had to write EIGHTY lines of example code to show how to set A SINGLE PARAMETER on SOME TYPES of cameras to its maximum value shows that THESE APIS ARE SERIOUSLY BROKEN. If you ever find yourself saying, "Look, to use these APIs I've written, you just have to write four tons of this glue code which never changes..." just stop! Write the damn glue code yourself, and provide a higher-level API!

--

You might hope that QuickTime for Leopard, which has been publicly announced to be at least partly rewritten with Cocoa APIs, will solve these problems. I certainly hope so. But, historically, some Carbon teams, when faced with writing Cocoa code, take a simple and not-very-helpful route -- they provide only one or two methods in their Cocoa classes, when their Carbon APIs are incredibly rich and feature-ful. The first iteration of QuickTime in Cocoa was like this -- look at NSMovie, which has TWO whole methods, "-URL" and "-QTMovie".

Wow! I'm overwhelmed! I mean, it's pretty clear how I would add a track, or record from a camera to a movie, or save to disk, or... Oh, wait, no, I can't do any of those things from Cocoa. I've essentially been given a busy-box. "Here, you're a Cocoa programmer, you can't handle actual functionality... just put your little movie in your little view in your little NIB and be happy."

But, wait, my scorn is too quick. Look what we got in 10.4, to replace NSMovie: QTMovie, which seems pretty darn rich from where I'm standing. At least, it has more than two methods, so I think that's a big improvement.

Or let's take NSSpeechRecognizer, written by a new friend of mine from WWDC, whom I'm sorry to single out this way.

Now, NSSpeechRecognizer has a nice interface. It's clear how to use a NSSpeechRecognizer, and it's easy as heck to integrate into your applications. I put speech recognition in Delicious Library 1 in about a day using this class, and most of the work was in building up a sane list of words to recognize and keeping it up-to-date without too much overhead.

But if you only speak Cocoa, you'll think that all Apple's speech recognition can do is what NSSpeechRecognizer does: recognize a flat list of words and phrases. You'd assume you can't recognize a phrase from one list followed immediately by a phrase from another list, much less set up a tree of expected phrases. And certainly there's no way to turn someone's speech into a list of phonemes for further processing, or to record and analyze the actual tones the user uttered. Heck, there are only four attribute set/get methods in NSSpeechRecognizer, so that must be about it for Apple's code, huh?

Well, as you've guessed by now, no. I'm told, by the original programmers, that all the above functionality is in the Carbon APIs. However, I have no idea, because I simply don't have the patience to learn or program raw Carbon unless I'm forced to, and in this case I wasn't.

There are a ton of neat applications I could make if NSSpeechRecognizer (and its friend, NSSpeechSynthesizer) were fully Cocoa. It's too bad.

--

Ok, let's pick on another Carbon function call, and then invent another Cocoa API that should exist but doesn't.

This time, it's from Keychain. Now, Keychain is massively cool; all your passwords, from all your programs, can be stored securely in a central (encrypted) location, so you don't have to remember them -- all you have to do is remember the password to your keychain. You get to set the policy on which programs can access which passwords and how. Keychain is full-featured and awesome...

...aaaaaaaaaaaaand it's Carbon. So, it brings all that baggage with it, which is really sad.

Let's look at some code I wrote recently to build up a list of FTP sites that are currently stored in the user's keychain, so I can automatically suggest a place to upload a user's Delicious Library in 2.0:

Build List of FTP Sites Using Carbon Keychain
#define HANDLE_ERROR(returnCode)if (returnCode != noErr) NSLog(@"keychain error %d encountered on line %d of file %s", returnCode, __LINE__, __FILE__);

- (NSArray *)allFTPSiteDictionariesFromKeychain;
{
NSMutableArray *mutableSites = [NSMutableArray array];

[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:kSecProtocolTypeFTPAccount]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:kSecProtocolTypeFTP]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:kSecProtocolTypeFTPS]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:kSecProtocolTypeFTPProxy]];

return mutableSites;
}

- (NSArray *)_ftpSitesWithProtocol:(SecProtocolType)protocol;
{
NSMutableArray *mutableSites = [NSMutableArray array];

SecKeychainAttribute findInternetPasswordsAttributes[] = {
{kSecProtocolItemAttr, sizeof(protocol), &protocol},
};
SecKeychainAttributeList findInternetPasswordsAttributeList = {
sizeof(findInternetPasswordsAttributes) /
sizeof(*findInternetPasswordsAttributes), findInternetPasswordsAttributes
};

SecKeychainSearchRef searchRef = NULL;
OSStatus returnCode = SecKeychainSearchCreateFromAttributes(NULL,
kSecInternetPasswordItemClass, &findInternetPasswordsAttributeList,
&searchRef);
HANDLE_ERROR(returnCode);
if (!searchRef)
return nil;

do {
SecKeychainItemRef internetPasswordKeychainItemRef = NULL;
returnCode = SecKeychainSearchCopyNext(searchRef,
&internetPasswordKeychainItemRef);
if (internetPasswordKeychainItemRef == NULL || returnCode ==
errKCItemNotFound)
break;
HANDLE_ERROR(returnCode);

#define HACK_FOR_LABEL (7)
SecItemAttr itemAttributes[] = {kSecDescriptionItemAttr, kSecTypeItemAttr,
kSecCommentItemAttr, kSecProtocolItemAttr, kSecAccountItemAttr,
kSecServerItemAttr, kSecPathItemAttr, kSecSecurityDomainItemAttr,
kSecCreatorItemAttr, kSecTypeItemAttr, HACK_FOR_LABEL}; // RADAR
3425797 - You can't get the 'kSecLabelItemAttr' using this API
(it crashes), so either we have to use the number '7' or use
SecKeychainItemCopyContent(). I do the former. See
http://lists.apple.com/archives/Apple-cdsa/2006/May/msg00037.html
SecExternalFormat externalFormats[] = {kSecFormatUnknown, kSecFormatUnknown,
kSecFormatUnknown, kSecFormatUnknown, kSecFormatUnknown, kSecFormatUnknown,
kSecFormatUnknown, kSecFormatUnknown, kSecFormatUnknown, kSecFormatUnknown,
kSecFormatUnknown};
NSAssert(sizeof(itemAttributes) / sizeof(*itemAttributes) ==
sizeof(externalFormats) / sizeof(*externalFormats),
@"arrays must have identical counts");
SecKeychainAttributeInfo info = {sizeof(itemAttributes) /
sizeof(*itemAttributes), (void *)&itemAttributes,
(void *)&externalFormats};

SecKeychainAttributeList *internetPasswordAttributeList = NULL;
returnCode = SecKeychainItemCopyAttributesAndData(internetPasswordKeychainItemRef,
&info, NULL, &internetPasswordAttributeList, NULL, NULL);
HANDLE_ERROR(returnCode);
if (internetPasswordAttributeList) {
NSMutableDictionary *siteDictionary = [NSMutableDictionary dictionary];
unsigned int attributeIndex;
for (attributeIndex = 0; attributeIndex <
internetPasswordAttributeList->count; attributeIndex++) {
OSType tag = internetPasswordAttributeList->attr[attributeIndex].tag;
if (tag == HACK_FOR_LABEL)
tag = kSecLabelItemAttr;

NSString *tagString = [NSString stringForOSType:tag];
id value;
if (tag == kSecPortItemAttr)
value = [NSNumber numberWithUnsignedInt:*(unsigned int *)
&(internetPasswordAttributeList->attr[attributeIndex].data)];
else if (tag == kSecCreatorItemAttr || tag == kSecProtocolItemAttr)
value = [NSString stringForOSType:
(OSType)internetPasswordAttributeList->attr[attributeIndex].data];
else
value = [[[NSString alloc] initWithBytes:
internetPasswordAttributeList->attr[attributeIndex].data
length:internetPasswordAttributeList->attr[attributeIndex].length
encoding:NSUTF8StringEncoding] autorelease];
[siteDictionary setObject:value forKey:tagString];
}
if (IsEmpty([siteDictionary objectForKey:[NSString
stringForOSType:kSecLabelItemAttr]]))
[siteDictionary setObject:[siteDictionary objectForKey:[NSString
stringForOSType:kSecServerItemAttr]] forKey:[NSString
stringForOSType:kSecLabelItemAttr]];
[siteDictionary setObject:(id)internetPasswordKeychainItemRef
forKey:@"keychainItem"];

[mutableSites addObject:siteDictionary];

SecKeychainItemFreeAttributesAndData(internetPasswordAttributeList,
NULL);
}
CFRelease(internetPasswordKeychainItemRef);
} while (1);

CFRelease(searchRef);

return mutableSites;
}

Now, as I said, Keychain doesn't have the worst interface in the world. It's really pretty good for Carbon -- you build the equivalent of an NSEnumerator with SecKeychainSearchCreateFromAttributes() (albeit without the nice generality of the former class), enumerate through it with SecKeychainSearchCopyNext(), and then get the data you want from the item in SecKeychainItemCopyAttributesAndData(). But at this point in this marathon post (aren't you sorry you picked on me for not posting in so long) you're an old Carbon hand, so you can spot the sins Carbon brings with it.

But, just in case, let's go over them:
  • We have to build up a bunch of structures by hand so we can build up a "search specifier" in a single parameter in SecKeychainSearchCreateFromAttributes(), which hurts our readability a ton -- we really want for each statement we write to read like a sentence, almost, and if we have to write four lines of "struct this" and "allocate that" before we get to the verb, we've got pretty awkward sentences.

  • We loop over the items we find using "SecKeychainSearchCopyNext()" (nice), but we have to manage memory ourselves in Carbon; there's no autorelease (or garbage collection, heaven forfend!) so we have to free every returned value explicitly.

  • The returned object is an array of structures of structures of void pointers to things with counters and stuff and things -- ok, I've totally forgotten, honestly. But, my point is, it's not just an NSDictionary full of nice objects (or a CFDictionaryRef), so I have to parse out the values by hand, INCLUDING byte-swapping on Intel machines. Whee! I love it!

  • SecKeychainItemCopyAttributesAndData() has a crasher if you ask to retrieve the most common attribute you'd ever ask for from a Keychain item: the label. I should point out that this function is documented to deprecate the older method of getting data from an item, which was more obtuse but didn't crash.

    Now, ok, I write crashers too. I don't blame the Apple engineers for that, and it was really nice of them to step up and provide a workaround. Because of their help, this only delayed me by a couple hours instead of a couple days. My point here is, if this whole framework were a lot more easier to use (-cough-Cocoa-cough-) then it'd get used a whole lot more, and this bug would have been found LONG ago. The problem here isn't that a bug was written, it's that an incredibly valuable framework is needlessly crippled by being in Carbon and thus not used enough to be as robust as it should be.

  • SecKeychainItemCopyAttributesAndData() isn't actually documented correctly, so the only way to write code for it is to get on the web and look at the source code and read Apple's mailing lists. In particular, the docs say for "attrList": "On input, the list of attributes in this item to get..." which is, frankly, just false. A look at the source code (search for "ItemImpl::getAttributesAndData(") clearly shows "attrList" is ignored on input, which only makes sense, because it's an output structure and you've already passed in the list of attributes to get in "info", so why would you pass it in twice? Note also the docs say that "itemClass" is "A pointer to the item’s class" but don't say if you are supposed to pass it in, or if it gets returned to you? (It turns out to be the latter). And "outData" is described opaquely as "A pointer to a buffer containing the data in this item." Oh, the data, you say? Well, that's certainly specific! I'm always getting and setting data on my items.

    It turns out that data in this case means the actual encrypted password. So, in summary, what the documentation for this function doesn't say, but should, is, "You build up a list of parameters you want to fetch from itemRef in info, and this method returns their values in attrList if that's not passed in NULL. The item's class (here there'd be a link to all the classes) is returned by reference in itemClass if it's not passed in NULL, and the password is decrypted from the keychain into outData if that is not passed in NULL."
Ok, we've rewritten the documentation, now let's rewrite my original method to use a hypothetical Cocoa Keychain framework. Note that this rewrite is longer than the last because the base Keychain APIs are more recent than the QuickTime ones I pimped above, so they already use more modern metaphors and are thus already somewhat efficient. But can we do better? (Hint: yes.)
Build List of FTP Sites Using Hypothetical Cocoa Keychain
- (NSArray *)allFTPSiteDictionariesFromKeychain;
{
NSMutableArray *mutableSites = [NSMutableArray array];

[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:NSKeychainSecurityProtocolTypeFTPAccount]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:NSKeychainSecurityProtocolTypeFTP]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:NSKeychainSecurityProtocolTypeFTPS]];
[mutableSites addObjectsFromArray:[self _ftpSitesWithProtocol:NSKeychainSecurityProtocolTypeFTPProxy]];


return mutableSites;
}

- (NSArray *)_ftpSitesWithProtocol:(NSString *)protocol;
{
NSKeychainFetchRequest *keychainFetchRequest = [[[NSKeychainFetchRequest alloc] init] autorelease];
[keychainFetchRequest setPredicate:[NSComparisonPredicate
predicateWithLeftExpression:[NSExpression expressionForKeyPath:NSKeychainSecurityProtocol]
rightExpression:[NSExpression expressionForConstantValue:protocol]
modifier:NSDirectPredicateModifier type:NSEqualToPredicateOperatorType options:0]];
NSError *error = nil;
NSArray *matchingKeychainItems = [[NSKeychain defaultKeychain]
executeFetchRequest:keychainFetchRequest error:&error];

if (error) {
NSLog(@"%@: Error doing keychainFetchRequest '%@'", keychainFetchRequest);
return nil;
}

return matchingKeychainItems;
}

_ftpSitesWithProtocol got so tiny because, assuming I didn't have to access the returned keychain results as raw data -- that is, if there were real, full-fledged NSKeychainItem objects that could be queried for their -protocol and -account and -password and -label, then I wouldn't need to do all the complicated object-building I did in the above example, I'd just return the actual keychain items, and even bind to them in my interface directly! This would be pretty rocking... you can imagine feeding an NSArrayController from this list and then binding columns in a tableView to "label" or what-have-you, and you've got a poor man's Keychain Access app in like five lines of code.

Now, again, this might be an oversimplification of Keychain's APIs. There may be subtleties that I'm missing that would require a slightly more complicated set of Cocoa classes. But in general, I think porting Keychain to Cocoa would decrease the amount of effort to program to it (and increase its audience) by an order of magnitude.

--

Let me wrap up, finally, by apologizing profusely to everyone at Apple and elsewhere whom I've insulted tonight. I've been a programmer for 25 years now, and every year I look back at the code I wrote the year before and think, "What the hell? What idiot-monkey wrote this crap?"

And yet, here I sit, on my medium-size pony, and insult code that some of you wrote ten years ago, viewing it through the lens of modern coding practices that you guys taught me. Let me stress this: the good programming practices I've learned, such that I've learned them, are from studying the very best of what Apple's done. There are tools and techniques that I love and am in awe of in Apple's code: bindings, enumerators, object-oriented database access, model-view-controller, all those buzzwords. I didn't invent 'em, you guys did.

I know that Apple people exist in the real world, where going back and rewriting blecherous code is not something any engineer wants to do, nor any manager to fund. I know if you meet with Steve in six months and say, "Hey, sure, we didn't add anything cool in Leopard for you to demo, but check it: Keychain is Cocoa now! Huh? Huh?" he'd probably put his foot so far... well, you know where I'm going with this.

And I know that it's a small miracle that things like QuickTime and Keychain and Speech Synthesis and Core Audio exist in the first place. These have truly amazing functionality, using research and math far beyond my ken: if they weren't, I honestly wouldn't give a crap that all their power is frozen in Carbonite right now. (You don't see me complaining about the, uh, Scrap Manager, do you?)

So, I write this to help you, putative Carbon-coding-Apple-engineer-reader, not to harm you. I want you to take this to your manager, and say, "Look, see! We've got to get to Cocoa! This has got to be a priority! We can't keep slapping tar on a boat made out of milk cartons!"

Don't do it for me, do it for your team. Do it because you'll love it, and because more programmers will use your stuff, and they'll show it off in ways you never imagined, and suddenly you'll be in all the Stevenotes, and you'll be wealthy and happy and famous.

Labels:

160 Comments:

Anonymous ssp said...

Don't you think it's a bit misleading to link the text 'a myth that persists' to a text from 2003? But that's probably just meant to be a snide remark towards the unsanity guys (who unlike many others are quite good at handling the Carbon stuff in their haxies).

Back then, this observation was true. Cocoa applications did feel a bit more sluggish than their Carbon counterparts. Time has moved on, Cocoa and the knowledge about it have improved, machines have become faster over these years and by now the speed difference vanished. Great for everyone!

Not that I'd disagree with any about the real points on Carbon and Cocoa you make. Trying the Mac Toolbox back in the 1990s was painful enough for me to give up programming. Just Cocoa made it possible for me as a casual programmer to come back and get the computer to do the things I want with a reasonable effort.

I definitely wish to see all of Carbon's features available through Cocoa, as many of them - just like your example - mean, that with the time and effort I can put into a fun project, I just have to do without them.

Hmmm, Cocoa Keychain...

October 05, 2006 4:16 AM

 
Blogger Wil Shipley said...

I don't write anything with the sole purpose of being snide towards someone. They wrote something I disagree with, I don't think it's snide to link to it.

I couldn't find the other link I was looking for, which was the much more recent announcement of some toolkit for Mac OS that billed itself as "having the speed of Carbon and the ease-of-use of Cocoa".

October 05, 2006 4:49 AM

 
Blogger Cameron Hayne said...

It strikes me that what you have said about some of the Carbon APIs is analogous to the situation in user-interface design.

A bad UI designer might put both "operationA" and "operationB" on the menu even though task analysis has shown that users will always do these two operations in sequence. A good designer would create a new higher-level operation that subsumes the two.

In more general terms, a bad API, like a bad UI, often results from a failure to take the time to understand what the users need to do, and/or a failure to make decisions about what should be included. ("We're not sure what they'll need so let's just put it all out there - make it flexible.")

Of course, a bad API, like a bad UI, can also result from an attempt to meet the needs of two (or more) very different groups of users in one product. (serving two masters)

October 05, 2006 5:32 AM

 
Anonymous [maven] said...

What scares me most is that the Keychain-bug is still unfixed and in the mean-time there have been 1125164 other Radar-entries...

October 05, 2006 5:34 AM

 
Anonymous Chris Ridd said...

So you think the security framework's horrible...

I would strongly advise against looking at the Directory Services framework, which is not only worse, but worse and inconsistent. :-(

October 05, 2006 6:58 AM

 
Anonymous Mark Lilback said...

I'm largely in agreement with almost everything you say. However, I think you're nuts complaining about FSRefs.

Paths are evil, way more than int constants versus enums. Fortunately, NSDocument internally uses FSRefss or aliases, so if I move an open document it saves to the new location, not where it used to be. Which is what you get when you use paths.

And FSRefs aren't hard to use. There are two functions to interchange between them and paths. Not even 10 lines of code to add a couple of methods to NSFileManager via a category. Same for aliases.

October 05, 2006 8:34 AM

 
Blogger Daniel Jalkut said...

FWIW there is already a pretty extensive ObjC wrapper on the Keychain, authored by one of Apple's (former?) engineers, Wade Tregaskis:

https://sourceforge.net/projects/keychain/

I haven't used it, but I noticed it while searching google for help in debugging my own, much less developed wrappers.

October 05, 2006 9:00 AM

 
Anonymous Anonymous said...

So what about all those Cocoa methods that take an NSDictionary full of parameters? It's really not that different.

And paths are definitely evil. About all you can say in their favor is they're cross-platform, so you don't behave any worse than elsewhere.

October 05, 2006 9:23 AM

 
Anonymous Anonymous said...

You’ve finally discovered that the QuickTime team at Apple is totally insane. We all have plenty of complaints about the APIs that live under the Carbon umbrella, but most have dropped Handles by now. Oh, but not QuickTime. They even have their own documentation site, carefully designed to place the information you need in three or four completely unrelated and separately-styled sections. Phew.

October 05, 2006 10:00 AM

 
Blogger Tristan O'Tierney said...

Great article Wil. I think ideally there should be some sort of free license utility kit of carbon-wrapping foo made/distributed by fellow mac devs like ourselves. Of course we don't live in a fantasy world where work is given out for free, but if someone started an initiative and publicised it well enough it might get *somewhere*. I know every time I deal with carbon I get the sensation that I require several showers, so I'd certainly contribute where appropriate if I ran into carbon snags.

October 05, 2006 12:25 PM

 
Anonymous fet said...

Be careful; "psyche" is a different word from the slang exclamation "psych!"

October 05, 2006 12:52 PM

 
Anonymous Anonymous said...

With respect to transitioning QuickTime to Cocoa, what about the Windows community? There seem to be a fair number of Microsoft developers hanging around and it would be interesting to see how Apple would accomodate them if they transitioned QuickTime from Carbon to Cocoa. Would they divide their energies equally to provide support for OS X AND Windows users? Yikes!

October 05, 2006 12:59 PM

 
Anonymous Andre said...

So this is what you've been doing all this time!
I've been waiting for another "Will Shipley" blog entry for a long while.

Well said.

October 05, 2006 1:06 PM

 
Blogger Aaron Tait said...

I think we (Cocoa Developers) should create our own open source collection of Carbon->Cocoa APIs. I'm sure that there are tons of developers, including me, that are writting this code. I don't know how exactly someone could implement this (Subversion maybe?), but there has got to be someone that is up to the challenge (i'm jsut a mere starving student). Who knows, maybe Apple would pick it up?

October 05, 2006 1:33 PM

 
Blogger Wil Shipley said...

Objective-C runs perfectly fine on Windows right now. There's no reason for the QuickTime team to avoid it.

In the past the QuickTime team has ported the entirety of the Mac OS Toolbox to Windows (and that port is now called "Carbon"); I don't think porting the support parts of Cocoa they'd need would be insurmountable.

October 05, 2006 2:15 PM

 
Anonymous Anonymous said...

It's funny you mention Core Audio. I don't know what it's like on their team, but on my team within Apple, we also have to support 3rd party developers who write plugins. And let me tell you that they do not want to have to learn or write that confusingly hard Objective-C stuff! (Aaaahhhh!! Brackets!)

In their defense, many of them need their code to be cross-platform and work in other applications in addition to ours, so I can understand wanting to do as little work as possible to deal with cross-platform issues rather than writing cool new plugins. But there is strong resistance to anything Objective-C in some of those areas. So I understand why certain technologies haven't added a Cocoa interface, especially when the development community for them is extremely small and opposed to using Objective-C.

October 05, 2006 2:29 PM

 
Anonymous Ian said...

Wow - it's like you can see inside my head.
I pity myself for not having the time/patience/wherewithall to have got this down myself.
Feel free to ask my rant-weary colleagues if you don't believe me.
Thank you Wil,
Much appreciation.

October 05, 2006 3:27 PM

 
Anonymous Anonymous said...

Don't you think the whole Carbon vs. Cocoa thing is a bit disingenuous? If you're going to go on the record and say that you have nothing against "modern APIs that happen to be written in C" then the problem with Carbon is that it's a bad API, not that its not Cocoa.

October 05, 2006 4:32 PM

 
Anonymous Anonymous said...

I'd like to make some points:

- Cocoa comes from chocalate. The brown delicious stuff that melts in your mouth. Apple did not, I repeat, did not invent this!!

- If i had to choose between Carbon or Cocoa, I'd choose Cocoa.

- About this objective-c character. I'd like to tell him one thing. You cannot be still on a moving train. You cannot be objective in a subjective world.

- Hi Fish-a-meal. You might want to switch it up. Maybe Fish&vegetables-a-meal. A little wine, some bread. Oh yea. Get drunk and watch batman.

- Chris, get ridd of that name. Tristan, change that name. Aaron, you need to stop wearing make-up. Fet, you missing an 'e' in the middle.

- Over and out Macdaddy's. Spend some time trying to get laid once in a while.

October 05, 2006 4:37 PM

 
Anonymous TIm Buchheim said...

I agree with almost everything here. The big exception is on FSRef vs pathnames. Pathnames suck, at least in the Mac world where users want to be able to move and rename files all the time. Not that FSRef is all that great .. we need a Foundation class with the same functionality.

And to be really smart, it could implement NSString's methods and automatically resolve pathnames when needed, so you could call old path-based API with it. (There could be problems with that .. I haven't thought it through completely, as it just popped into my head, but it could be cool.)

Toll-free bridging with FSRef would be icing on the cake.


At least we're reaching parity on the GUI stuff (although a few widgets are still only in one or the other) although NSViews and HIViews still don't play together the way we'd all like them to do. (Every release has moved us closer to that, however small those steps might be.)

I think QTKit and Core Image are both good indicators that Apple is trying to move towards making it possible to access everything via Cocoa but I imagine it takes a lot of careful consideration to wrap functionality in ObjC without ending up with half-assed solutions like NSSpeechRecognizer.

October 05, 2006 5:04 PM

 
Blogger The Nog said...

If you're going to go on the record and say that you have nothing against "modern APIs that happen to be written in C" then the problem with Carbon is that it's a bad API, not that its not Cocoa.

Cocoa is useful for comparison because of how high-level its APIs are. It's a handy example of what a "good" Carbon should strive to be rather than requiring so much low-level glue code. Avoiding that stuff is part of the reason I embraced Cocoa in the first place.

October 05, 2006 5:57 PM

 
Anonymous Anonymous said...

means reading a function description doesn't actually give you enough information to know what to actually pass it

Dude, welcome to C. I can't imagine what would happen if you had to program for Win32. Your head would surely implode.

October 05, 2006 6:50 PM

 
Blogger Peter Hosey said...

In your remodeled _ftpSitesWithProtocol: method, did you forget to use the “protocol” argument? (And I presume that it should be re-typed as NSKeychainSecurityProtocolType and the calls in the method above should be changed to pass NSKeychainSecurityProtocolTypeFoo.)

October 05, 2006 6:52 PM

 
Blogger John Siracusa said...

I can't wait until Wil graduates to a managed, dynamically typed language in 15 years and writes a big article about the barbarism of Obj-C and the Cocoa APIs. Come on in, Wil! The water's fine! ;)

October 05, 2006 7:07 PM

 
Blogger Wil Shipley said...

I can't imagine what would happen if you had to program for Win32.

Nobody has to program for Win32. That's kind of the point of my life.

October 05, 2006 7:15 PM

 
Blogger Wil Shipley said...

John: I can't wait, either. When I see a set of APIs that's better than Cocoa I'll switch to them. I'm not going to switch languages without the APIs, though.

October 05, 2006 7:16 PM

 
Blogger Wil Shipley said...

Peter: thanks for the pointer. Should be better.

October 05, 2006 8:22 PM

 
Blogger Wil Shipley said...

So what about all those Cocoa methods that take an NSDictionary full of parameters? It's really not that different.

It's not good, but it's certainly better. The nice thing about the Cocoa methods is that NSDictionary is a standard class, so it's easy to build up -- you can do it in one line, and in fact you can do it in-line if you don't have a big dictionary to build.

Since they are autoreleased, you don't have to declare them, set them up, pass them in, and tear them down. You just say:

[myThing setParameters:[NSDictionary dictionaryWithObjectsAndKeys:@"orange", NSThingColor, @"blue", NSThingBackgroundColor, nil]];

And you're done.

However, you still don't get type-checking and it's not self-documenting what -setParameters: takes, so this is broken like Carbon in those ways, but at least the Cocoa documentation always has a reference to where you can find the parameters for their monolithic set/get methods.

It's sometimes a good idea to do monolithic set/getters (eg, when setting up styles in an NSAttributedString, it makes sense, because you often want to re-use the same groups of styles over and over, so you'll set up the dictionaries once and hold on to them), but in general I think it's a bad practice under Cocoa and discourage it.

But it's certainly not the same as Carbon.

October 05, 2006 8:27 PM

 
Anonymous Tim said...

For Chris Ridd: For what it's worth, I did an Obj-C translation/extension of Apple's C++ DSWrappers sample code for Directory Services a couple years ago. Disclaimer: There are known problems in the code, but it works well enough for what I needed to do that I've never got around to dealing with them.

October 05, 2006 10:55 PM

 
Anonymous Anonymous said...

Only 80 lines of code? You Mac programmers have it good. You should try calling COM stuff from .NET once in a while :-)

October 06, 2006 12:14 AM

 
Anonymous Anonymous said...

I enjoy your hilarious implication that Carbon dates from 1980 and Cocoa from 1997. The facts are that Carbon was rebuilt from the ground up in the late 90s, and all it shares with the original Mac toolbox is some APIs and basic ideas, whereas Cocoa is the direct continuation of OpenStep which dates from 1993. Yes, Cocoa is significantly older than Carbon.

Another thing I enjoyed about your post was your very strong implication that silent failure is a good way to deal with errors. Your code must be incredibly fun to debug.

Anyway, keep up the amusing posts.

October 06, 2006 12:16 AM

 
Anonymous Rosyna said...

There are a few major issues with your "argument".

1. Complaining about FSRefs pretty much completely invalidates the entire thing. Paths suck, plain and simple. Paths put the control of the user's computer and files on the running application. FSRefs put the control in the user's hands.

2. This entire article is prefaced with the idea that it's about carbon. Yet you don't mention a single carbon API other than an opaque type, FSRef. QuickTime isn't carbon, the Security framework isn't carbon, Core Audio isn't carbon. The "closest" thing to carbon you mention are the Speech APIs. The rest are just procedural C APIs. The QuickTime team is a completely separate entity all together. You can easily determine this as they made simple bugs in HIMovieView that no Carbon engineer would make.

Hell, none of the specific functions you complained about even existed in Mac OS 9. Again, the closest thing was FSRefs, which were added to the Mac OS in Mac OS 9, but the finder and other things didn't use them so they didn't offer much benefit.

3. Virtually nothing in carbon returns a FourCharCode. Well, excepting GetEventClass() of course. Many APIs take FourCharCodes as it allows a method to create a unique int value that means something and is mostly human readable. After all, FourCCs are a standard (http://en.wikipedia.org/wiki/FourCC).

4. You say Carbon is a legacy API, but if you took a program written for the Mac toolbox in 1990, and tried to compile it for Mac OS X, it would not compile. Massive amounts of changes would be needed to make it modern. On the other hand, take an OpenStep program and try to compile it, most things would work except those that used frameworks that have better replacements today, such as MediaKit. Cocoa is the much, much older and legacy API in this case.

5. Your problem with the 80 line sample seems to be with error checking. Are you saying it's a bad idea to check for errors, recover from those errors, and present the user with meaningful information on how they can resolve the errors? All of your code either has the calls failing silently or throwing an exception, which leaves the program in an undeterminable state which can (and usually does) lead to a crash. Are crashes better than proper and correct error checking?

6. I stand by the slow speed assertion. Most Cocoa programs are slower than the carbon counterparts. Look at Backup.app, iPhoto, Delicious Library, Mail.app, and others. They all slow down immensely when given huge sets of data. Huge being relative, of course. Most of the speed issues aren't really caused by issues with cocoa itself, but with the mentality cocoa creates. The belief that cocoa will do everything for you. This causes the programmer to believe there is no reason to optimize. So you've got some extremely expensive calls such as creating an NSDateFormatter, iterating a directory, et cetera being done every single time a row in a table row is drawn. When the same formatter and the same iteration could be cached and used for every single row. KVO and things like bindings just make this worse and it makes so the programmer has to write even less code, which leads to slower code.

7. Also, it seems to be that everything you complain about seem to be personal difficulties with the APIs, not a failure in the API, just your personal approach to using them. A simple example is the fact most C APIs on OS X that are documented are documented mostly in the actual header files. Whereas the cocoa documentation is all in HTML files and the ilk. FourCCs are probably one of the easiest things to look up. Imagine trying to search all the headers for what 'rgnh' means in any given context compared to what the int value 7 means.

October 06, 2006 12:51 AM

 
Anonymous Anonymous said...

As if someone who accidentally leaked a full serial number generator with his application, and a very ugly one to boot, would have so much authority on what makes for good or bad code!

October 06, 2006 1:13 AM

 
Anonymous Michael Stroeck said...

Rosyna, you don't seriously think that Mail.app, iPhoto and Delicious Library are too slow for more than 0,5% of the users? Nobody single piece of software will ever be fast enough for the power users, so that argument is not very convincing.

Besides, that is not the point! Applications such as iPhoto and Delicious Library and the vast majority of modern independent Mac software very definitely would NOT EXIST in their current form and feature-richness if there were no high-level APIs to work in.

October 06, 2006 1:46 AM

 
Blogger Wil Shipley said...

Wow, the Carbon trolls are out in full force. I hardly know where to start.

First off, I'm not going to argue what is and isn't Carbon. If you want to define Carbon to be twelve men reading the Torah, that's fine. Whatever you decide we should call the set of procedural C interfaces that encompass QuickTime and Keychain and Core Audio and Speech, well, that's what I'm talking about. Don't care about the name.

And if you guys seriously don't know the difference between handling errors and writing tons of crappy code for no reason, there's not much I can do for you. I posit that I should be able to call -setGain: on any camera, whether it has gain or not. If you disagree, that doesn't negate my argument that Cocoa-style APIs would be better -- you could just add a single call like "-(BOOL)gainControlIsAllowed;". Now we're up to TWO lines of code. GASP!

My point isn't that there shouldn't be error-checking, it's that Apple should do it for us, and just return one status. Hell, if they want to raise an exception that says, "Sorry, no camera" or "Sorry, no gain on this camera" or whatever, that's fine. But I shouldn't have to write the exact same 80 lines of code every other developer does.

Also, seriously, Apple's sample code error-checking sucks. They never do anything constructive with the errors. Most of the sample code just says, "Here's where you'd handle the error!" Sometimes they log it to the console. Oh, wow! Good thing you trapped that! But what, pray tell, should I *do* if the sequence grabber shuts down with return code -2124, which isn't documented anywhere? Stand up? Sit down? Fight? Fight? Fight?

I'd like to mention I didn't attack you or you guys' damn apps in my post, and I think it's pretty classless when you to attack me and mine. It's the lamest form of logical fallacy to attack the speaker instead of his argument. ("Gee, you write some pretty ugly code for someone who is saying that George Bush tortures people!")

But, hey, since you brought it up, let's talk about Delicious Library, ok, Rosyna? Yes, it's slow if you load in hundreds of items. Version one was not designed for that many. Version 2 is designed for tens of thousands, so customers with larger collections will be happy there.

Yes, I made design decisions so I could ship my app. Surprise. I decided to concentrate on making a beautiful and functional and stable app, at the expense of it being slow for large collections. I wish I could have done EVERYTHING in version 1, but I simply couldn't. I made a trade-off. Big fricking deal.

Are there lots of optimizations I could have made to Delicious Library 1? Yup, you bet. But, for most of the project, it was just me coding. Seven months, start to finish, brand-new app. We were living on our savings, I had no severance pay from Omni (just a used laptop), and I had to ship. So we prioritized what we thought was important, and we did it.

But, I'm going to state this: I've never seen any application by a startup company sell as well in its first year as Delicious Library. Period. I've never seen any third-party Mac app sell this well in its first year. Period.

Sure, they may be out there -- I don't have everyone's sales data. Maybe Unsanity is selling a zillion copies of your apps every day, I don't know. But I'm pretty damn happy with the success of Delicious Library, and it was pretty much written by me.

So I think I did OK. I think my priorities are NOT wack. I think speed is great, but it's obviously not the _most important_ factor to most users. Cocoa isn't inherently slow. It does allow you to write code very quickly, and some of it might be slow. The nice thing is, you can choose which things need optimizing, and which are fast enough.

Look, it comes down to this: if you think Delicious Monster is a success and you'd like to see how I did it, I write about that here, in this little blog. Go ahead and read. If you think my code sucks and I'm full of myself, well, hey, guess what... DON'T FUCKING READ MY BLOG!

Seriously. Guys. I don't force you to come here and read my opinions. I take time off work to write up things I feel strongly about so I can help people out. If you disagree, fine, but don't attack my shit. It's just not cool.

As for the publishing my key generation code -- let's talk about that. Yes, I had to figure out some ugly Carbon APIs to generate keys, and yes, it was ugly. A lot of the reason it was ugly was because I was trying to be clever and do all the checking in inline functions, so that a hacker couldn't just NOP a single function or method in my code and do an end-run around security -- the security code would actually be injected into the main app in several places.

However, I used a framework (legacy!) for part of the application, and in linking to this framework I forgot to check off the "don't export headers" button, so all the headers (including the one containing all the ugly licensing code) got published in version one of Delicious Library.

Yup, that's right. Gasp. If you're a pirate, you can generate keys to Delicious Library. You can steal my software, much like, oh, every other piece of software in the known world. Guess what my attitude is towards that? If you guessed "yawn", you're only partly right.

It actually turned out to be a huge win to publish my licensing code, because this way the hacking groups on the net went to the trouble to generate their own illiegal license keys instead of stealing a legitimate customer's keys.

So, with each point release of the software we do, we can just look on the net for the latest illegal keys and shut them down -- if they were keys owned by legitimate customers, we wouldn't want to shut them down, because we're not Microsoft (-cough-WindowsGenuineAssholes-cough-). Now, some of those people are going to just go and get another illegal key. All right, fine, some people really aren't going to pay for software. When I was 12 I stole software too, I don't think the economy died because of it.

But SOME people are going to say, "Damn it, this is cool software, I don't want to keep screwing with using pirated keys that will get shut down all the time," and they're going to go ahead and purchase Delicious Library.

What makes me so sure? Last time we did a minor release, which ONLY fixed one minor bug (focussing control for external iSights on Mac Pros), we did an EXTRA $12,000 of business that day and the next. Cash money business. Not $12,000 total, an EXTRA $12,000 above our normal business.

So, yes, if you'd like to keep flaming me with an oversight I made two years ago regarding header files and key generation, go ahead. But I can't help but feel I got the last laugh here.

October 06, 2006 1:58 AM

 
Blogger Wil Shipley said...

The facts are that Carbon was rebuilt from the ground up in the late 90s, and all it shares with the original Mac toolbox is some APIs and basic ideas

I feel like what I'm talking about here are archaic APIs, not how old the code is. What I said was its heritage is in the 80s, which is a statement you didn't refute in the slightest.

But I'm glad I amuse you. Keep laughing!

October 06, 2006 2:02 AM

 
Anonymous Anonymous said...

Seven months, start to finish, brand-new app.

So despite the articles to the contrary, the developer mentioned in your about box, and the upgrade pricing, Chronopath Library didn't help jumpstart DL's development at all?

October 06, 2006 2:32 AM

 
Blogger Wil Shipley said...

Bait bait bait. I don't talk about Chronopath out of respect for the deal we struck.

If you've got a problem with me, why don't you come out and tell me what it is? It's obvious you want to get me upset. Did I do something to you? Hurt you in some way? Maybe I can make it right.

October 06, 2006 2:44 AM

 
Anonymous Anonymous said...

My problem is simple: you constantly state that you singlehandedly built DL from scratch in seven months, when the reality is that you upgraded an existing product, which is a much simpler job. Whether you say this to make yourself look better or for some other unknown reason is not something I can say, but it is very annoying to see someone with so many fans acting that way.

October 06, 2006 2:48 AM

 
Blogger Wil Shipley said...

Actually, I didn't say that -- I said I worked mostly alone.

You don't know the percentage of code from Chronopath Library that remained in Delicious Library the day we shipped it, do you? No, you don't. Guess who does? Me.

As well, you don't know what percentage of code that was written by the owner of Chronopath Library that made it into Delicious Library 1.0, do you? No, again, you don't. Again, take a guess who ran those numbers? Who did a long subversion check, figured out the percentages.

Any guesses? Any at all? No? Ok, it was me, again. I know, you don't.

I don't respond when people say that we based our app on a teenager's work because it really wouldn't be cool to talk about that whole deal. He's a good kid, he's got a good future.

You can believe what you wish about me, but if you don't LIKE being angry, I'd ask you to consider that you may not have all the facts, or even some of the facts, and that there may be a reason I'm not giving those facts which is not purely selfish.

October 06, 2006 3:03 AM

 
Anonymous Anonymous said...

You just proved me right. The percentage of code that remained is irrelevant. I've worked on more than enough code to know that starting from a stable base makes life much easier, even when none of the base remains at the end of the day.

So say what you like, and yes I don't have all the facts, but by your own word I know that DL was not "Seven months, start to finish, brand-new app." So why do you insist on saying that over and over again?

October 06, 2006 3:32 AM

 
Anonymous Anonymous said...

Very interesting article indeed. To my opinion, it stresses the importance of SW architecture as you are mainly comparing high-level calls (cocoa) with low-level ones (carbon).
OK, this sounds like an overall simplification of the issue, but it still strikes me to see that despite the thousands of documentation pages on developer.apple.com, there is no real overview of the MacOS X SW architecture.
I wish we could have a nice and layered block diagram, going further than Application Services on top of Core Services (as presented in the Getting Started section of Cocoa).
Wil's QT example is very typical in that respect: 80 lines of code due to mixing low-level calls for high-level purpose (setGain:)...

Regards, Arnaud

October 06, 2006 3:57 AM

 
Anonymous Anonymous said...

The problem you' re presenting doesn't seem to be with "Carbon" vs Cocoa at all. It's with C APIs vs ObjC APIs. I'll posit that you won't ever meet a commercial, successful C API that you will like, at least judging from this post. Because to obtain the features you seem to like, the API designers had to move on to an OO language.

The Quicktime example is just an example of how Byzantine the Quicktime API is. Everyone agrees. And lots of "Carbon" engineers find it just as difficult to read. We all agree that Quicktime isn't as easy to use as it could be -- even Apple. If you look at some of the newer stuff they are doing exactly what your asking. Like the audio extraction APIs. Setup some parameters for the extraction, ask for audio, do what you like with it. It takes care of the details. Although it doesn't use dedicated property getter/setters and it doesn't use dictionaries everywhere, it's still a great API for getting sound from formats that Quicktime understands.

And speaking of that, there is a reason that Quicktime generally doesn't have dedicated property getters/setters. Because you can have some new object (file type, hardware, decoder, etc) that knows how to do something that Quicktime itself has no clue about! The programmer can know about the new property without Quicktime having to have documented it anywhere, rather than waiting for an API to expose that property. For some applications (such as the types of file that Quicktime can access) the full list can never be fully documented, because it is ever changing and specific to each user's machine. If I wanted to present to a user the file formats that they could save to, I could either hardcode a list (which is bad) or enumerate the list of available ones (which is good) but the latter is always going to be more complicated, no matter how simple a veneer is put on it.

And lets talk about what is Carbon and what is Cocoa for a moment. You know what Apple generally considers Cocoa? AppKit, Foundation and CoreData. That's it. Every other ObjC API in the system is just that -- and ObjC API. That doesn't mean they might not use meteaphors from Cocoa, but as others have pointed out most modern code on the system uses those metaphors too (to the extent possible at least).

So what's Carbon then? This one's harder. The lines aren't nearly as clear because a lot of Carbon has been extended to be the basis of extensive system functionality. The major reasoning being that Carbon is a C API, thus there has never been any issue calling it from Cocoa (C++ had issues early on, but we're past those now). One is tempted to define Carbon as the entire C API on the system, and you would be completely wrong. Calling IOKit a Carbon API will get you smacked by some people, as just one example. And there are a lot of C APIs on the system that Apple didn't write. That pesky open source thing. There is also a lot of code written specifically for Mac OS X. By your own definition of Carbon they couldn't possibly be part of Carbon because they never existed on Mac OS 9. Your most amusing example is CoreAudio -- CoreAudio is complex by definition, it's an API for exposing hardware in a uniform way. But unless you care about the hardware itself you shouldn't be using it. Instead use the AudioUnit or AudioToolbox frameworks, they do more work for you.

But we still haven't defined Carbon. I bet if you ask this question on Carbon-Dev you'll get a different answer from every developer. Carbon is an overused Marketing Term (just like Cocoa is) at this point. The only thing that is even remotely exclusive is HIToolbox vs AppKit for creating UIs. That's it. And you can even intermingle them in the same application. And who knows, someday you may be able to mix them at an even finer level. And the line gets even fuzzier.

So what do you define as Carbon then? If you define it as a set of APIs, then I'd define it, at maximum, as HIToolbox, CarbonCore, and maybe CoreFoundation. That's it. If you define it as the clients of those frameworks as well, then almost everything is Carbon. Yea, no one is gonna by that one. But then, many Cocoa folk seem to believe that everything that is a client of the ObjC Runtime is Cocoa. Which makes every application a Cocoa application. Isn't this so very confusing?

So in the end, what does it matter? File some bugs requesting simpler alternatives if you don't like some APIs. But framing it in terms of a Carbon vs Cocoa your implying that there is a systemic issue here when there isn't. Some APIs are good, some are bad, some are new, some are old (some are DIRT old), and some use old metaphors, some new ones. Your going to get a culture shock whenever you move from an API that uses a metaphor that your used to to one that doesn't. It doesn't make the API bad, it just makes it different.

October 06, 2006 4:28 AM

 
Anonymous Anonymous said...

Well, it seems that Apple recently updated its view on MacOS X architecture as it now provides a good block diagram with the different elements: see Mac OS X System Architecture

From this page, you'll notice that Quicktime is one level bellow both Carbon and Cocoa... Maybe the idea that everything in C must be Carbon while objective C is Cocoa needs to be refined!

Regards, Arnaud

October 06, 2006 5:11 AM

 
Anonymous JS said...

There's a response to this post on Daring Fireball, but it's not very good. No offense to Gruber, but he sort of completely misses the point of this post so that he can point out that Carbon and Cocoa both date back to the 80s, even though you didn't actually call one "old" and one "new".

October 06, 2006 7:25 AM

 
Anonymous Jonathan Wight said...

So Wil,

These 80 lines of IIDC code? Should Apple be spending engineering resources providing a nice OO replacement for Carbon/QT code that already works and is used by very few applications? Or should be using their resources to add things like Spotlight, CoreData, CoreAnimation, TimeMachinet, etc to Mac OS X?

I know where I'd rather Apple use their Engineers.

October 06, 2006 8:55 AM

 
Anonymous Anonymous said...

Amen, Wil.

Let me add that during my time at Apple, I saw one group after another realize that if they made their technology easier to use, then more people would use it (and they'd be in a better position to compete for staffing, funding, etc.)  The whole company is moving that way, but some parts will of course do it faster than others.  

One big part of the problem with the persistence of Carbon however, is that the squeaky wheel gets the grease, and the most numerous and loudest complaints that came into DTS and WWDR in general were from the luddites.  That's why HIView exists, for example.  Yes, it was necessary to let OS 9 code run outside of the Classic environment, but Carbon should have been in maintentance mode from then on.  As heroic an effort as it was, there was no need for HIView, period.

Now, DTS's chain of command does not consist of ex-NeXT developers, and they frankly do not realize the difference in productivity between the two environments, so they went a bit overboard in coddling the foot-draggers. 

The way I think of this is that the organization acted as an enabler rather than as a therapist:  the patients suffer more if you allow them to drag out their addiction than if we'd just told them flat out: NO, you can't get all the new tech into your out-of-date application.  Sorry, but you'll really be better off if you chop your 300K lines of Mac Toolbox code down to the 30K lines or so it would take to re-implement using the modern approach. 

Hell, you should have seen how tough it was to get some people to even give up using Quickdraw, for god's sake.

-jcr

October 06, 2006 9:43 AM

 
Anonymous Anonymous said...

For TIm Buchheim:

There is an NSString subclass for FSRefs, sort of. Check out NSPathStore2. It's private, though.

October 06, 2006 10:19 AM

 
Anonymous Anonymous said...

I agree that the keychain API is pretty nasty, but I was able to dumpster-dive what I needed from Apple's sample code. Fortunately I've never had to delve into QuickTime, but I've had to use AppleEvents to get/set Finder comments that are lost by all file copy/move API, and it was a hideous experience. The newer CoreFoundation APIs are pretty nice to work with, though (I say this as someone who learned Cocoa first).

Re: the FSRef vs. path issue, though, paths have some serious limitations, in spite of their convenience and presentability. Apple has NSFileLocator and NSFileSpecifier (according to MagicHat), but they're private. I use BDAlias for storage on disk, and wrote an FSRef wrapper to work around path/NSURL comparison problems: BDSKFile (Disclaimer: work in progress). The problem is that CFURL apparently doesn't do a nonliteral comparison (or whatever is correct for paths), so the results vary depending on the Unicode normalization form of the path used to create the CFURL (which varies depending on the API used to get the path). The path strings have the same problem.

October 06, 2006 10:56 AM

 
Anonymous Anonymous said...

So I understand why certain technologies haven't added a Cocoa interface, especially when the development community for them is extremely small and opposed to using Objective-C.

Perhaps the community would grow if you offered a Cocoa interface...

October 06, 2006 11:51 AM

 
Blogger Drew Thaler said...

Oh, c'mon. The fact that you're agitating for more Cocoa wrappers is well taken. Apple is slow to prioritize wrapping stuff, agreed. Part of that is that in a lot of cases third parties have stepped in and done the work for them already. (eg, Daniel Jalkut pointed you at one for the Keychain.)

But you're just distorting things with your deliberately-chosen examples. Let's take a look at those "80 lines of code" for IDC, shall we?

Lines of whitespace: 10
Lines of comments: 20
Lines of error checking: 7
Lines with continuations of an expression/function solely for readability: 15
Lines with nothing but curly braces: 12

That's fifty-four lines, or 67%, devoted to making the code understandable and useful. Call me silly, but that's AWESOME for sample code. Whoever wrote this in DTS deserves a raise, not criticism.

I mean, I'm sure they could release a special crappier version of the example just for you. Would that make you 67% happier? :-)

October 06, 2006 1:46 PM

 
Blogger Drew Thaler said...

Also, since I am a former Apple employee who's helped create both Carbon-like and Cocoa-like APIs (for DiscRecording) I can actually explain some of the stuff that's going on here.

High-level vs low-level: Frameworks and features -- and therefore APIs -- are necessarily written from the bottom up. You can't write a high-level API without knowing what's in the low-level API today and what is likely to show up in the future. If the low-level API is a new technology and still subject to change, it often makes sense to delay creating a high-level API for it. If there are only enough engineering resources to do one of the two, you get one guess which it will be.

Difficulty of creating a high-level API: You argued this quite well with your comments saying that in a Cocoa API, stuff should just be taken care of for you, values should be normalized, all the details and errors hidden, etc. Well, sure. That's great when it's somebody else's problem. Heck, my computer should just do all my work for me! :-) But designing a good API that actually DOES that takes time. I would estimate that it takes roughly a year per major subcomponent to get to a good design that actually "does all that stuff for you" with no major problems. (Yes, it took about a year for DiscRec to develop our own ObjC API.) That's a lot of engineering time.

Deprecating APIs: APIs are only deprecated; they are almost never truly deleted. C functions usually can't be deleted because of the requirement for backward compatibility -- even if the implementation is replaced by a stub, the symbol must remain. So it's not as easy to make stuff go away as you imagine.

Giant attribute calls: Carbon-like C APIs do this deliberately; it's partly because (as already explained) deleting APIs is not easy, and partly because of symbol table size and link speed. Think about what would happen if every object in every framework started exporting 50-100 different calls for 50-100 different attributes. Multiply the number of attributes by the number of objects in all the different frameworks in the system. Then multiply THAT by 2 or 3 because everything is fat now. The symbol tables you need suddenly go beyond "huge" into "big enough to have their own moons". And we're not talking about short symbols like "setGain" ... because C means a flat namespace, you have to prefix everything. You wind up with something like "QTSequenceGrabberCameraIIDCControllerSetGain". ObjC not only uses shorter symbols, it is able to avoid this problem in other ways. Carbon-like C APIs cannot. So they intelligently push it off onto the app developer, rather than slowing down the whole system.

API vs implementation: The low-level API is usually intimately connected to the actual implementation. Down at that level you really want the more efficient and correct implementation -- not the nicer API. eg, using a "nice" float value may actually make it impossible to specify the exact gain value that the hardware needs.

Source code vs compiled size: Despite your pokes at the amount of text in the source code, it actually compiles down fairly nicely.

Apple being slow to create Cocoa wrappers: Most definitely. Those jerkoffs, they never make stuff like that a priority. Apple sucks.

But we all knew that already. :-)

October 06, 2006 2:34 PM

 
Blogger Wil Shipley said...

Should Apple be spending engineering resources providing a nice OO replacement for Carbon/QT code that already works and is used by very few applications?

Well, yes, I think I just wrote a huge post saying that. I guess I failed.

My point was, more applications would use this code, in incredibly innovative ways, if it had higher-level APIs.

I know where I'd rather Apple use their Engineers.

Ok, well, you're not alone in that, but I respectfully disagree.

October 06, 2006 2:41 PM

 
Blogger Wil Shipley said...

All: I absolutely agree there should be an object-oriented file class to replace paths. Handling paths isn't fun.

I just think FSRef isn't it. In part because it's object-oriented, and part because I find it hard to use since, through no fault of its own, it wasn't really designed for long paths, Unicode paths, and UNIX filesystem conventions.

I certainly understand that there would be some loyalists to FSRef while we wait for the Foundation team to give us an NSFileReference or some such.

October 06, 2006 2:46 PM

 
Blogger Wil Shipley said...

That's fifty-four lines, or 67%, devoted to making the code understandable and useful.

I agree to a point, but part of my point is that QuickTime code needs that extra stuff. It's my general practice to count whitespace when you count lines, but I understand if you don't... there's still a huge collapse here.

Call me silly, but that's AWESOME for sample code. Whoever wrote this in DTS deserves a raise, not criticism.

It is, in fact, exceptionally good example code, and I am sorry if it seems I'm taking shots at the DTS person who wrote it. I am trying to take shots at the fact that it needed to be written at all.

DTS people should be dealing with such high-level APIs that they can spend their time writing cool little apps that show how the technology can be used in new ways, and not hot to set gain control on IIDC cameras.

And I didn't choose this example because it's degenerate; I chose it because I had to do it myself and as far as I know only one other developer in the world(!) has done it as well.

And, yet, I can think of lots of applications where having camera input and controlling things like its focus would be incredibly useful and fun. I don't think there are only two of us because QuickTime isn't useful; I think there are two of us because QuickTime needs powerful high-level APIs, in the form of Objective-C.

October 06, 2006 2:53 PM

 
Blogger Wil Shipley said...

Anonymous flame guy:

You are right. You win. I am a bad person and I hereby renounce my fame and ask everyone not to pay attention to me.

-Wil

October 06, 2006 3:00 PM

 
Blogger Wil Shipley said...

After reading John Gruber's response over at Daring Fireball (and not figuring out how to post comments there), I think I it wasn't clear enough what my point was when I gave the origins of Cocoa and Carbon.

What I was trying to say was, "The authors of Carbon aren't idiots, they are just patching a framework that is using older metaphors, and the results are always going to be unacceptable to me because I want newer metaphors."

I wasn't trying to say, "Carbon is old, Cocoa is new, thus Cocoa is better," but instead the more subtle, "Carbon is based on an older metaphor of programming and is inherently flawed in my view because of it."

There. I wish I'd written just that and skipped the whole post entirely, actually.

October 06, 2006 3:21 PM

 
Anonymous julian said...

Wil,

I appreciate that people take the time to post their disagreements and that you take the time to respond to them (however inflammatory they may be). I learn just as much from the arguments as I did from the original post.

Thanks everyone!

October 06, 2006 5:39 PM

 
Anonymous Anonymous said...

Will, you trumpet your business success with Delicious Library. Fine, great.

But then you say you're not MS and because you're not MS you're not going to kill off publicly-released Serial Numbers? WTF?

Look, it's basic business. If a customer releases their serial number to the world, either maliciously or accidentally, the smart, honest, ethical thing to do is cancel the serial number. Easy.

So the customer gets back to you: waaaaah! My serial number don't worky any more!!!

So what? You then decide how to deal with that individual customer (i.e. giving them a replacment serial with a warning, or tell them to bugger off, or ask them for their ICBM coordinates, or add their email address to a fetishist mailing list, etc.). In the meantime, you help protect your business and legitimate customers aren't screwed at all.

October 06, 2006 6:19 PM

 
Anonymous Anonymous said...

Wil, Thanks for the article. And thanks to the people who wrote the constructive/substantive comments. I learned a lot.

As for people posting the personal attacks, get a life. Several people made me rethink my views about non-Cocoa APIs without doing the nasty dance. Nobody looks good when they're dancing the nasty.

jd

October 06, 2006 6:31 PM

 
Blogger Lucas Newman said...

But then you say you're not MS and because you're not MS you're not going to kill off publicly-released Serial Numbers? WTF?

Actually, what he said was we do disable pirated license codes that have been published, but we aren't going to extraordinary lengths to prevent piracy with activation schemes like Windows Genuine Advantage. We don't want to punish our honest, paying customers, even if that means we lose a couple of sales.

October 06, 2006 7:03 PM

 
Anonymous James Gregurich said...

Drew Thaler said...

High-level vs low-level: Frameworks and features --...


Why are we writing low level procedural C APIs rather than object-oriended C++ or ObjC low-level APIs? Many issues raised here would be resolved if a proper object oriented design was used from the ground-up rather than high-level OO wrappers being put around a low-level C API.

-James

October 06, 2006 8:28 PM

 
Anonymous Anonymous said...

Wil writes an example of an inline dictionary:
[NSDictionary dictionaryWithObjectsAndKeys:@"orange", NSThingColor, @"blue", NSThingBackgroundColor, nil]

I've only tried Cocoa in passing, but the thing that grated on me the most was its verbosity. Sure, it's a fraction of your carbon examples, but your inline dictionary would look like { ThingColor:'orange', backgroundColor:'blue' } in Python.

October 06, 2006 8:34 PM

 
Anonymous Anonymous said...

Drew,

I didn't take Wil's post as any kind of knock on whichever DTS engineer wrote the sample, but rather as a complaint that the antiquated API in question required that much code.

-jcr

October 06, 2006 9:14 PM

 
Blogger Drew Thaler said...

Why are we writing low level procedural C APIs rather than object-oriended C++ or ObjC low-level APIs?

I can only answer with my personal opinion, as I'm no longer with Apple, but:

C++ is out because of fragility. Mac OS X lacks a solution like COM, and thunking and padding classes with dummies has turned into a big pain in the ass in the few places (IOKit) that it's used.

ObjC is out because -- like it or not -- it requires a whole runtime, and there is a huge amount of code inside and outside the system which is not ObjC and doesn't have that runtime. Everything in the kernel. Every existing app which made Mac OS X feel like a Mac back in 2001. Unix tools and daemons. And so on. If we'd written DiscRec in ObjC only, there wouldn't have been any CD burning in iTunes or the Finder. Ultimately all this stuff, taken collectively, is a big part of what makes OSX into OSX rather than NeXTstep 2006.

C APIs are the remaining common denominator. Everyone will freely admit they suck, but everything sucks in some way or other. They are at least well-understood, work for everybody, and hold no surprises.

October 06, 2006 9:30 PM

 
Anonymous cesar said...

wil,

I think everybody knows about library coming from chronopath, WHO CARES? what is important is that DL is a great app. Does anyone b$%ch about FinalCutPro been bought by Apple from Macromedia? or Is DVDStudioPro a good apple gone wrong just becuase apple got a small software company? MS claims they develop DOS and the real stuff is that they got it from someone else and make DOS a sucess.

I just wish I could have the balls to get my own company and be "THE MAN" and not work for "THE MAN".

I enjoy reading your opinion about software, but they are just your opinions and personal attacks on you and your company are not allowed or fair.

i'm just waiting for DL 2.0 and other DM apps.

just remember to have fun while codin'

October 06, 2006 9:59 PM

 
Anonymous James Gregurich said...

C++ is out because of fragility.

I see no reason why that can't be dealt with by the appropriate use of subclassing, namespaces, and versioned frameworks. Perhaps someone can educate me on that why these three tools wouldn't suffice to work around C++ fragility.

YOu can encapsulate data members in C++ classes the same way CF does if necessary.

there is a huge amount of code inside and outside the system which is not ObjC and doesn't have that runtime.

Cannot the authors of the Finder and Itunes put C or C++ wrappers around a ObjC API? What's the difference if a pointer returned by a procedural API C function is a pointer to an opaque structure or a pointer to an NSObject subclass?


-James

October 06, 2006 10:19 PM

 
Anonymous Bret said...

Hmm... interesting post. However, I (predictably) disagree with a few points...

1) C is faster than Cocoa (and I do mean Cocoa, with all it's associated high-level-baggage, not just Obj-C), partly because it's "legal" and "accepted" to take certain optimizations that are frowned upon when writing OO code (like directly accessing structure members), and partly because there's no runtime thunking going on to resolve calls (although, the obj-c runtime is pretty good at what it does).

Now - does this mean that every app should be written in pure C? Or even assembler (where you can take even more optimizations)? Heck no... much of your code only runs infrequently, and on doggon quick hardware, at that. But, for some things (especially where you are pushing the envelope), you need the speed (I happen to be working on just such a project now - dealing with many gigabytes of data structures, all of which must be accessed "simultaneously" - the overhead of building that with Cocoa constructs like collections and using KVC to get at it is prohibitive). Similar patterns occur in low-level frameworks like Quicktime and OpenGL - you need the speed, because you are (or at least, were - remember the days when playing back a full-screen video was really impressive?) really pushing things.

2) Even when code must be portable to other systems, there's no reason why there can't be a clean Cocoa interface to it. I've implemented a toll-free-bridge in that huge-dataset project that I'm working on, and if you are building the relevant module(s) on something without obj-c, it just compiles (obviously, you will only have the C api available in that case). Operations on the dataset that iterate over the whole thing are written in C, and accessible from obj-C as well... and of course, you can throw any bridged object in a collection or do any other obj-C thing to it...

3) There are actually three developers outside of Apple that are interested in the camera APIs that I know of - yourself, Bonix, and me... but I was looking at the APIs in question back when the iSight was first released, and I hit a brick wall in trying to find the setting for accessing the focus attribute (I was trying to get it to focus about 3 in away from the front of it - the camera will do it if you monkey with the sliders in the settings pane, but I couldn't find the API to set it and get it to stay locked). Upon hitting said brick wall, I moved on to other projects... but would like to investigate going back to this... hmm --- would it be possible for you to post the section of your code that sets up the iSight? :-)

October 06, 2006 10:49 PM

 
Anonymous Wade Tregaskis said...

Nothing new said here, but well said, and very true. I must say though, that ultimately you need to put your money where your mouth is, so to speak; I've written my ObjC wrapper (http://sourceforge.net/projects/keychain/) around the Security framework more or less by myself, so the important questions are:

1) Why aren't you using it?
2) Why aren't you contributing to it?

You've taken the time, Wil, to write a neat little bit of example code for how you'd like to do things; why not take a bit more to extend my Keychain framework to do what you want (specifically, the bindings and predicate parts)?

October 06, 2006 11:04 PM

 
Anonymous Simone Manganelli said...

With regards to the whole error handling debate, it doesn't really have anything to do with Shipley's argument. Handling errors is as simple as adding an extra line of code. Instead of:

return [sequenceGrabblerChannel setGain:1.0];

you could simply use:

NSDictionary *errorDict;
return [sequenceGrabberChannel setGain:1.0 withErrorDict:&errorDict];

One extra line of code, not 87 extra lines of code. This is precisely how NSAppleScript objects do error handling. The framework could even pass user-readable error messages to the referenced error dictionary, and the developer could simply pass that on to the user in a dialog.

Even though Shipley doesn't use error handling in his example, it doesn't mean that it can't be done in a simple, effective way.

Besides, the whole debate over the precise definitions of "Carbon" and "Cocoa" are likewise irrelevant (as Shipley himself pointed out). Call it "old metaphors vs. new metaphors" if you wish, but it doesn't change the fact that using frameworks like QuickTime before QTKit were a pain in the ass. I've had personal experience with QuickTime as well as Authentication Services, both of which are unnecessarily complicated for Cocoa developers.

October 07, 2006 1:33 AM

 
Blogger Wil Shipley said...

Wade:

You raise some good points. Primarily, my mouth didn't know where your money was before I made this post, so I figured I'd be alone. But, yes, many times I've discussed with my crüe writing wrappers for this or that Carbon library.

There are a couple things holding me back. One is, it's a ton of work writing and maintaining frameworks. I don't really have that kind of time right now; I'm trying to get an app out the door that's like six months overdue in my customers' heads (to be fair, I've had a ton of free upgrades up to now, but those don't really count in most people's view).

Two is, I've vowed never to write code that I don't FULLY use myself, because I think it leads to buggy, crappy code. Apple is in a position to do it because they've got thousands of people to test they stuff, but I don't have that luxury. I learned from hard experience at Omni that writing general frameworks when all you need is some specific functionality is a fool's errand. (Many people disagree with me. That's OK.)

Three is, I don't know that I could get Apple to adopt my code; I've had mixed results writing example code in the past. At Omni the Ken wrote a general-purpose networking framework that's pretty good, but even though we offered it to Apple for free they wrote their own. (Which has its charms, but initially wasn't nearly as full-featured as OmniNetworkstuff. Mind you, I use CFNetwork all over the place now, so far be it from me to argue.)

On the other hand, the PDF-parsing technology I wrote for Jonathan Schwartz's old company (before Sun) was purchased again by Apple for CoreGraphics, but even that code, I was told later, was just used as a rough reference by the guys implementing PDF in CoreGraphics, as they had to do their parsing in C / C++ and had to implement a whole new rendering engine on top of it. (And my hat is off to them -- you guys knocked it out of the park. CoreGraphics r0x0rz my b0x0rz and escapes my C-hating wrath because it is just damn fine stuff. Not that I would appreciate some damn Objective-C toll-free bridging, Graffagnino. Yah, I'm looking at you, pal.)

I've learned from, say, writing web browsers, that I don't like competing with Apple directly when they give their stuff away. It's a waste of my effort. I don't mind giving back to the community (that's why I've sponsored things like XP on Mac and the Gimp for Mac effort), but I want to make darn sure my efforts aren't going to waste.

Finally, four is, I'm not easy to work with. The first thing I'd do if I got involved in something like the SourceForge Keychain is reformat all the code and rewrite it according to my conventions. It occurs to me this kind of thing might not fly when working with people who aren't my employees -- even my employees kind of hate it.

All that said, I may still do it, because I'm just that dumb. But I don't know that I'd start with Keychain. I mean, the Speech stuff needs love too. And LaunchServices? Don't get me started.

-Wil

[By the way, I should point out, for those who are STILL arguing about what is and isn't Carbon, I mostly take my cues from Apple: (http://developer.apple.com/reference/Carbon/index.html) Is QuickTime on there? Yes, yes it is. So, Apple calls it Carbon, Apple invented the word Carbon, I guess I'm going to call it Carbon.]

October 07, 2006 1:43 AM

 
Blogger Wil Shipley said...

Bret:

I hit a brick wall in trying to find the setting for accessing the focus attribute [...] would it be possible for you to post the section of your code that sets up the iSight? :-)

This really pains me, but right now I'm not releasing this code. The reason is that my direct competition is also claiming they can scan barcodes from iSight cameras, and one of (what I estimate to be) my three key advantages over their approach is that I know how to focus an iSight, which I've never seen done before outside of Apple (or, heck, even inside).

Sad, right? But it took me a week to figure this stuff out, and when I'm in business for myself, I don't like giving a week's worth of effort directly to my competitors. (It's not even like they'd credit me for it.)

There are obviously some cool things you could do with manually focussing a camera, like taking a single picture over and over again ten or so times, with a different focus every time, and then running some software on each frame to figure out which parts were in focus and which were not (there's a heurestic to do this that I discovered when researching image deconvolution), and then creating a single in-focus composite image that also has a rough z-buffer on it.

Yes, I'm talking real 3D photos -- and since it's just z-buffering, you could slide two of them together and move them around in z relative to each other to interleave elements correctly (imagine taking two diaramas and merging them together). It probably wouldn't be perfect, but it'd be fun as heck to play with.

-Wil

October 07, 2006 1:50 AM

 
Blogger Wil Shipley said...

Also:

C is faster than Cocoa (and I do mean Cocoa, with all it's associated high-level-baggage, not just Obj-C), partly because it's "legal" and "accepted" to take certain optimizations[...]

You can do these in Objective-C, since you can always drop down into C. There's a specific example I can think of but not talk about, but if you were at WWDC this year you probably saw it -- it's FAST.

October 07, 2006 1:52 AM

 
Anonymous Andrew Welch said...

I posit that I should be able to call -setGain: on any camera, whether it has gain or not. If you disagree, that doesn't negate my argument that Cocoa-style APIs would be better -- you could just add a single call like "-(BOOL)gainControlIsAllowed;". Now we're up to TWO lines of code. GASP!

How would this be any different than if there was a Carbon framework that had a function:

SetCameraGain(myCameraRef, 1.0);

?

The answer is that it wouldn't be any different. You can design an excellent API in Carbon, and you can develop a truly lousy API in Cocoa. There are numerous examples of both.

Carbon vs. Cocoa is so 2002. We do both Carbon and Cocoa; both have frustrating limitations. The fact that obscure, poorly documented, neglected APIs that grew organically suck is no surprise to anyone.

That's a far cry from being able to make broad-brush generalizations like "Cocoa good, Carbon bad". What you're really saying, whether your realize it or not, has nothing to do with Cocoa or Carbon at all.

That is to say, what you're really saying is that Apple should provide excellently written, full-featured, documented APIs. I agree they should, but they don't.

For as many obscure "mystical cookie" nonsense as there is in things like QuickTime, there are just as many Cocoa classes that fall off of a cliff after a few basic methods.

October 07, 2006 2:24 AM

 
Blogger Wil Shipley said...

SetCameraGain(myCameraRef, 1.0);

The answer to your question ("How would it be different") is:

- I couldn't extend the myCameraRef object in your example, because it was a brittle C pseudo-class.

- It wouldn't return an NSError or throw a standard exception if it had a problem, because it would be at the Carbon level, so it'd return an OSType like every damn Carbon call does.

- I'd be in charge of freeing the myCameraRef because I wouldn't have autorelease or even, someday maybe, garbage collection.

- The parameters would not be self-documenting because the calling syntax is C.

- It would be harder to read because C's functions essentially read as "verb(subject, object)" instead of a more English-like "[subject verb:object]".

This is much, much worse in the multi-parameter case. For instance, guess what this function does:

SetCameraModes(myCameraRef, 1.0, 0.0, TRUE, FALSE, up, down, diddly);

Can't do it, can you? What if it were:

[myCamera setFocusDistanceInInches:1.0 gammaCorrection:0.0 autoCorrectImage:TRUE autoFocus:FALSE directionOfSky:up directionOfEarth:down thingFlandersSays:diddly];

Xcode autocompletes. There's no cogent argument that can be made against "wordiness" unless you type like a N00B or you love APL. (If you love APL so much, why don't you marry it?)

--

Seriously, Objective-C good, C bad. Cocoa has advantages like having a dynamic object model, being able to subclass system classes, having somewhat decent memory management, and even uncomfirmed rumors of garbage collection coming to Objective-C.

Carbon doesn't cut it. It's bad. Bad bad bad. There are ways to write better Carbon, but there aren't reasons to do so. It should go away.

If you love old interfaces, go ahead and use them, just don't ask Apple to make them better. I can't imagine both being a luddite AND expecting Apple to continue to enhance the old technology you covet. It's like me saying, "Look, I hate cars, but I also hate going slow -- why can't you damn engineers stick a turbocharger up my horse's ass?"

October 07, 2006 2:46 AM

 
Anonymous Pierre Lebeaupin said...

You are about to enter a war zone. Go on anyway (y/n)? y
Password:*******

So, Wil, you took on what I hoped you would never speak about, because it's my main disagreement with you. Carbon. As I said earlier when I blew up your first sub-event loop example in the previous Pimp My Code (which I like very much and are very informative), I'm more of a Carbon guy. But definitely not, I hope, a Carbon troll. And yet I must comment on some things I think are debatable.

First, Carbon is not a compatibility library. Perhaps it should have been (there had been some wondering about that at the time following the announcement of OSX, in 1998), but it is a subject that's much too big to be discussed here. It is a full-fledged environment inherited from Classic MacOS, just like Cocoa is inherited from NeXT. But, since contrary to Cocoa, it had never been meant to run in a modern, memory-protected, preemptive-multitasking system, it had to be modified extensively to run under OSX. Which brings me to the following point: these changes were NOT minimal. Many apps never got ported. And ask any Carbon developer what the Carbon transition has been like.

I don't think "the early 1980s/in 1997" is fair comparison either, but indeed Carbon is definitely older that Cocoa, there's no denying it. More importantly than age itself, Carbon did not age as well as Cocoa, it's true. This is mitigated by the fact most parts that didn't age well have either been unsupported with Carbon, or replaced with more modern equivalents. But of course some pervasive standards remain (though you're lucky, I think you don't have to use any such thing as a Pascal string to program in Carbon nowadays), such as OSTypes.

(btw, I think Apple has always recommended Cocoa for developers starting out in OSX AFAIK.)

As for the speed myth, Andrew Welch already dispelled it for me back then in this article, long before I got OSX myself. As he says, "you can write a slow MacOS X application in either Carbon or Cocoa. You can write a fast MacOS X application in either Carbon or Cocoa". I don't think there are fundamental differences of speed between the two (except maybe for stuff that's going to be pretty much negligible in the end, such as messaging vs function call, and even then messaging's dynamic binding may be really meaningfully used, which means there would have to be more than a mere function call to provide the equivalent service), but developers may tend to be less attentive over what they're actually doing (keeping stuff in memory, not caching API results, overusing messaging) with Cocoa, I don't really know. Still, some things in particular may be a problem, as in this post.


Now, on the subject on what is Carbon and what is not Carbon. Stricto censu, I'd say that are in Carbon the managers/services in the Starter Kit and the Expansion Pack in there, because they form a coherent whole where the parts depend on each other, or at least assume you're using the others, and are the pretty much required basis for building an app. You can also see Carbon as the modified subset of 6000 API (out of the 8000 in OS9) that have been selected for being ported to OSX, plus the ones that have been added to these 6000 to replace some of the 2000 missing ones, plus maybe the APIs which directly integrate to these 6000+ and have been added later (say, HIStuff); on top the stricto censu list above, this includes QuickTime (which is a world in and on itself), and legacy stuff such as the "Classic" Event Manager and QuickDraw. And of course you can also see Carbon as everything that presents a C interface (as opposed to an Objective-C one), and isn't in Darwin (BSD Unix stuff and the like), which makes it pretty darn big, more so that Cocoa (taken as everything that presents an Objective-C interface). I would rather talk of APIs presenting a C interface (C APIs in short) vs APIs presenting an Objective-C interface (ObjC APIs in short). Of course these C APIs borrow things from Carbon, but whatchagonnado? Since they present a C interface, they cannot make use of Objective-C/Cocoa objects, types, or memory management, to name a few, so they have to use conventions coming from Carbon (and I'm sure you would hate them even more if they used conventions coming from Unix/POSIX; ever checked out ioctl?). (last note: one could also consider Carbon everything that's part of the Carbon umbrella framework, though I don't really know what it amounts to).

The other way round, there are a few ObjC APIs without any procedural C equivalent, such as Core Image or Synch Services. Not that I want an equivalent procedural C API to be provided, I am able to use ObjC if need be without problem. Notice there is an inequality here: Cocoa programmers can call C code without problem from their .m files, while Carbon programmers will have to write wrappers in .m files or switch entirely to objC source files altogether (which technically shouldn't be a problem, but may explain part of the resistance) to use Objective-C interfaces, not to mention setting up autorelease pools and other such environment stuff.

Now, on to some more specific things. First, not all parameters are passed by reference, only structures (or of course function results other than the error code, but you don't have much choice for them). And I'll say that passing structures by reference is a Good Thing, because passing them by value means copying them for little point since they're not going to be modified by the called function, using more stack space to make room for the copy, and throwing other parameters out of the zone of parameters that are passed in registers. Now, these passed-by-reference-because-they're-structure params should be declared as const more often, so that this fact is at least more self-documenting.

However, such getters and setters, anonymous params and other such things are indeed very annoying, and on top of that badly documented. Sometimes it looks like an excuse not to document them (hey, we documented the functions, what do you need now?). And documentation in headers is NOT satisfying at all, for obvious and less obvious reasons. We're truly far from the wonderful Inside Mac documentation, where I learned most of my Carbon knowledge before much of it became obsolete. But interestingly, ObjC methods are selectors. Pretty darn self-documenting, parameter-explicit, compiler-checked, etc... selectors, but this means they don't take up space in symbols tables and slow down linking, so one can afford ad add many of them.

As for OSTypes, well, I like them. Yep. When you don't have automatic (Java's garbage collection) or semi-automatic (Cocoa's autorelease) memory management, that's the best thing you can have for tags since they fit in a 32-bit value. Now I agree they suck as selectors, but we're just coming back to point 1 and 2. Handles and atoms are going to be found only in QuickTime in fact, you (as a Cocoa programmer) will never need to use Carbon stuff that uses handles, and atoms are QT-specific. As for FSRefs, paths are the evil. Teh suck. The antechrist. And FSRefs HAVE BEEN designed for long paths, Unicode paths, and UNIX filesystem conventions. They have been introduced in MacOS 9 (i.e. they were not in MacOS 8) to go with HFS+ (as opposed to plain HFS) for that very purpose, beforehand there was FSSpec which does not support these AT ALL.

About your two code examples, yes, it's hair-raising, especially with QuickTime. Yes, having to build up structures and arrays by hand explicitely and then use them is less readable. But I think that these wonderfully functional expressions give a false sense of efficiency, and do not encourage caching results or reusing the same dictionary without rebuilding it each time, and other such practices. Now, I know that I should not optimise as I go, because I should leave optimising to after everything is complete and Shark has proven that's where I spend the most time, but still all this unoptimised code is going to have an impact eventually.

My problem with this blog item is that it's not so much Carbon vs Cocoa as low-level vs high-level API. For instance, Keychain is part of Darwin (which means it's not in Carbon by any extent), of course it does present a low-level interface, what else would you expect from something that's in Darwin? Things like that have to be accessed by many applications at a low level, because they need fine control over the thing, generally they're written is such a way (say, using the BSD interfaces) that it's more straightforward to use such a low-level interface where you admittedly have to do many things yourself; in fact some programs, which are particularly likely to use keychains, have critical security needs and/or use SETUID or other such stuff which mean they can't link with high-level frameworks, even Foundation, much less AppKit (or Carbon for that matter). There NEEDS to be at least a low-level interface for this service.

Now, what you would like is an elegant Objective-C wrapper over Keychain. But, doing such a thing takes time, the elegance of Cocoa libraries does not come from nothing, and I suppose some deal of time is needed only to make sure the object metaphor that should go over the "low-level" stuff is good, even before the actual, clean code can begin being written. Not to mention they have to wait for the low-level API to have stabilized before even starting doing so. And of course, they have to do so without compromising the security of Keychain.

This is even more the case for the advanced features (those that are not in NSSpeechRecognizer) of speech recognition. Building up a good object-oriented wrapper, using the right metaphor, over these complicated features (which is what you're looking for) is a hard and huge work. Of course in the end it's easier for you, but this means it has been proportionally harder to implement.


But my biggest problem is that throughout this post you have a derogatory attitive towards Carbon ("frozen in Carbonite", "boat made out of milk cartons"). Please show a little respect for these interfaces. Sure they are harder to use than Cocoa, but only slightly more when you're used to them. Sure it's more wordy (and I agree with counting the comments in the source lines, since one has to provide them to make up for the lack of self-documentation of the code), but many developers are going to need the fine control that's provided and would have to be as wordy (or more) using an ObjC interface to do what they actually want, there is no such thing as automagic (you're probably guessing by now that I'm an Ambrosia fanboy). Plus, they can do it RIGHT NOW instead of having to wait for Apple to have implemented a nice ObjC API around them.

I happily admit I've not touched Cocoa much (currently doing the Currency Converter with Bindings tutorial), and the few (Free) projects I am involved in are in Carbon partly because they use the Resource Manager (yes, that stuff before nibs and bundles that's limited to 2727 resources and 16MB of data in a single file), and I was familiar with Carbon even before I started programming. I also admit that as a student supported by my family, I can afford to write code at the pace I want and write many source lines if need be and ship whenever I feel like it: recently I updated one of my apps (whose main work is decoding images from a special run-length encoded compressed format in resources) to be Intel-native, and even though it compiled and ran perfectly (without any performance problem) on my MacBook once I got it (as I had plenty of time to add byte-swapping beforehand), I decided to partly rewrite the decoding code for Intel processors as I had initially written it unrolled four-way in parallel, which is good for PowerPC procs but means much stack spilling on Intel procs due to the limited number of architectural registers; and anyway shortly therafter I wrote AltiVec and SSE2 version of the decoding algorithm which basically rendered these changes useless since all Intel Macs have SSE2. I've even written a few routines in 68k assembly for running under the 68k emulator of the Classic environment, when I still had a PowerPC Mac, just for the fun of it. All this was interesting learning, but it's obviously not something I could afford to do otherwise.

I have no problem with you not liking Carbon, just like you don't like frameworks nor unit testing. We don't have the same needs. But please at least show some respect.

lebpierreÀwanadooPOINTfr

P.S.: Andrew has entered the steel cage! Ack! Run for your lives!

October 07, 2006 4:25 AM

 
Blogger Wil Shipley said...

Pierre,

Thanks for your well-reasoned post. I am afraid it is an almost irrepressable facet of my personality that when I dislike something I REALLY dislike it, and I show it very obvious disdain. This earns me enemies, but I've founnd half-measures convince no one. Sometimes we need firebrands because our fires need poking.

Essentially, I've never heard anyone defend Carbon (or low-level C interfaces) with anything other than, "We're more or less stuck with it, so stop whining." Well, I've heard people defend it, I guess, but I've never agreed with any of their arguments.

Yes, I think Objective-C should be part of Darwin. I think the performance is there, and there's no reason for anything to be written in C any more. It's not like we didn't have Objective-C device drivers for Intel machines back in the OpenStep for Intel Processors days, and processors were a HECK of a lot slower then.

To be a Mac programmer is to embrace change when it's for the good, and to eschew convention when convention is wrong.

I simply believe that Objective-C is a revolution. AppKit is a revolution. It's something I dedicated my life to when I was 19 years old, and have worked on ever since.

Try to imagine me, 18 years ago, telling people with a straight face, when they asked me what I did, "I'm going to destroy Microsoft." And when they asked how, I'd say, "I'm going to convince the world that NeXTstep is the right way to program, and we're all going to be better off for it."

Even my Dad thought I was crazy, then. As I've mentioned, now he's asking me if he can borrow money to help him buy a bigger boat.

So, yes, I'm a zealot. Like most zealots, I believe very strongly that I'm right. Unlike most zealots, I know that as soon as I win this war I'm going to switch sides. I'm going to find the Next Great Revolution and I'm going to fight Apple with all my heart.

Because it's not about Apple vs. Microsoft or Obj-C vs. C or Cocoa vs. Carbon or anything like that. It's about progress vs. fear. It's about embracing change for the good, or fearing all change.

Yes, I know I sound like a crazy person. Hey, I admitted right in the top line of my blog that I'm crazy. It may be the least of my personality flaws, even.

-Wil

October 07, 2006 5:16 AM

 
Anonymous Pauli Ojala said...

Hi Wil,
you wrote in a comment reply:

"Objective-C runs perfectly fine on Windows right now. There's no reason for the QuickTime team to avoid it."

It's true that Obj-C works on Windows if you use GCC / MinGW. But of course practically all commercial Windows developers use the Microsoft compiler which doesn't have Obj-C support. GCC and MSVC object code doesn't play nicely together either (e.g. struct layout differences), so it would be a pretty terrible idea if Apple made QuickTime pure Obj-C and just told Windows developers to use GCC...

Re: what exactly constitutes Carbon, I think the easiest definition is to simply look at Carbon.h. It's prefaced "master include for all of Carbon" and includes the following frameworks:

CoreServices, ApplicationServices, HIToolbox, CarbonSound, OpenScripting, Print, NavigationServices, CommonPanels, HTMLRendering, SpeechRecognition, SecurityHI, Ink, Help, ImageCapture.

October 07, 2006 6:23 AM

 
Anonymous Jim Stead said...

Will, consider yourself lucky there still exists a cocoa. I was at a meeting at Apple with technical management fairly soon after the Return of Steve where we were told objc was going away. Glad it didn't!

Jim

October 07, 2006 10:07 AM

 
Anonymous James Gregurich said...

1) C is faster than Cocoa (and I do mean Cocoa, with all it's associated high-level-baggage, not just Obj-C), partly because it's "legal" and "accepted" to take certain optimizations that are frowned upon when writing OO code (like directly accessing structure members), and partly because there's no runtime thunking going on to resolve calls (although, the obj-c runtime is pretty good at what it does).

I'm not exactly sure what the point is to this post. YOu can certainly directly access class data members in OO code if you so desire. Its just a bad idea to do it because it causes maintainability and extensibility issues in your code.

As for the "runtime thunking", you don't need to use that stuff where it causes performance problems. You can isolate areas where performance is critical and use a procedural approach if that is faster.

procedural code isn't inherently faster than object oriented code. However, if you want to implement a very maintainable and extensible design, your job will be a lot easier with an OO language than with procedural C.

-James

October 07, 2006 12:24 PM

 
Anonymous Anonymous said...

"Look, I hate cars, but I also hate going slow -- why can't you damn engineers stick a turbocharger up my horse's ass?"


Best Shipley Quote Ever!

-jcr

October 07, 2006 1:10 PM

 
Anonymous Rosyna said...

Procedural code is inherently faster than Objective-C code. There's no way to dispute this, really. The Objective-C runtime is highly and extremely dynamic. C code is static and unchanging. Because C is static, it is faster as there's no runtime mucking going on making the language dynamic. Whether the difference is 1ms or 4 seconds depends on the code and the working data set.

Wil, what Carbon functions return an OSType? Based on the context, I do not think it means what you think it means.

October 07, 2006 1:12 PM

 
Blogger David said...

Cheers to that, Wil. Since our discussion about sequence grabber at WWDC student day, I've discovered the ABOMINATION that is NewMovieFromProperties. "There's got to be a better way", right? -- david

October 07, 2006 1:39 PM

 
Anonymous Johan Kool said...

Hi Wil,

Gosh, you sure touched a sensitive issue apparently. But I think you are totally right in what you say. As a programmer in Cocoa of course we prefer a Cocoa way to access functionality. And yes, if for certain functionality we need to cross over to Carbon that is a very real barrier for many. Your appeal for Cocoa-way of using some basic things like KeyChain and QuickTime really isn't out of line. I respect that it's not a small effort for Apple to write this, but that doesn't mean it wouldn't be useful to many programmers.

All the Carbon programmers here, please do use Carbon as you prefer. Programming Cocoa I know that I would be much more tempted to use a functionality when I can access it easily. Only crossing over when I reallly reallly reallly reaaallly need too... only if I am reallly reallly reallly reaaallly hungry I'd try to figure out how to order a taco in Spanish. :-)

Take care you all! Don't fight to much! :-)

Johan

October 07, 2006 1:43 PM

 
Anonymous Anonymous said...

By the way, I should point out, for those who are STILL arguing about what is and isn't Carbon, I mostly take my cues from Apple: (http://developer.apple.com/reference/Carbon/index.html) Is QuickTime on there? Yes, yes it is. So, Apple calls it Carbon, Apple invented the word Carbon, I guess I'm going to call it Carbon.

Wil, you do realize that that page is virtually the same as http://developer.apple.com/reference/Cocoa/index.html. For one, Quicktime is listed on both pages.

Face it, most of the system isn't Carbon. And most of the system isn't Cocoa. Trying to force everything into one or the other definition is a terrible way to present the issue. There are high level and low level APIs in the system. Some of them will be good, some extraordinarily good, and some bad, and some, of course, extraordinarily bad. No getting around that. But when you assume that no C API can be good (when that it obviously false, at least to the community) and any proper ObjC interface will be good (again, obviously false) is when you lose respect.

October 07, 2006 2:46 PM

 
Anonymous Bret said...

Rosyna is right about the procedural vs OO speed thing - and I have firsthand experience that supports it. doSomething(foo *someFoo) is always faster than [someFoo doSomething] when [someFoo doSomething] translates to a function call, a hash lookup, a cache hit, and then another call. In a tight loop with a big dataset, it adds up (particularly with something that goes thru lots of accessors - and it's even more extreme when you are comparing foo->attr to [foo attr], instead of getAttr(foo *someFoo) to [foo attr]).

James Gregurich -- the point was that doing it the "Cocoa way", going thru accessors, using the provided collection classes, etc, is slower than doing it the "C way" - direct access to structures, and direct function calls. Yes - maintainability, readability, and robustness may all suffer; for raw speed does come at a price! But, when you are pushing the envelope, the choice is between doing it at all, or not doing it; not between doing it elegantly or inelegantly.

Yes - objective C does **allow** you to get down & dirty with your objects (I have a Core-Data alternative that I was working on that did just that to provide transparent ivar archival) but doing this violates convention, and is frowned upon. What I was pointing out was the speed difference between "The Cocoa Way" and "The C Way", not what could and couldn't be actually **done** in either language.

>> You can isolate areas where performance is critical and use a procedural approach if that is faster.

Yes! And some frameworks (like QuickTime, or that large-dataset project that I'm working on) are such places that I would, and (for my current project) do. But, there are high-level things that need to be done in my project as well, and for that I've toll-free-bridged my C stuff to my Obj-C stuff; so I get the best of both worlds.

>> procedural code isn't inherently faster than object oriented code.

Unless you break a few OO conventions regarding data encapsulation and abstraction, yes it is -- because the "OO way" is to generalize whenever possible, and the "procedural way" is to write to the specific case; and unless your general case happens to line up with the specific case 1:1 you (or rather, your computer) will be doing extra work to account for situations-not-at-hand.

>> However, if you want to implement a very maintainable and extensible design, your job will be a lot easier with an OO language than with procedural C.

Oh, and don't I know it! More than once I've looked at what I'm typing in C and said, you know, that'd be a lot more elegant in Cocoa... but shoot, I'm already whipping this box for all it's worth, and then some! And DANG I wish Apple would speed-bump it already!

Wil - I understand about that whole not-wanting-to-give-your-competetors-an-edge-thing... after all, a man's gotta' eat, ya know? FWIW, ***I*** wouldn't bother to try to write a competitor to DL, because, well, 1) it's already been done quite well, and 2) it just wouldn't be... um... delicious, ya know? ;-)

>> You can do these in Objective-C, since you can always drop down into C.

**Can**, yes - but I wasn't talking about what the language could do, but (as I mentioned earlier in this comment), about the "Cocoa Way".

>>For instance, guess what this function does:
>>SetCameraModes(myCameraRef, 1.0, 0.0, TRUE, FALSE, up, down, diddly);


Oh, that's simple: it does diddly! 8-P

Hei-diddly-ho! ;-)

October 07, 2006 2:52 PM

 
Anonymous James Gregurich said...

C code is static and unchanging.

yes. and that is very often not a good thing.

The problem is once you start trying to hack procedural C code when you need it to be dynamic, you make a huge mess that will likely be slower than just using the OO features of an OO language.

I work on a large legacy project that is entirely procedural C Carbon code. it is dog slow and a maintenance nightmare because it attempts to be dynamic, flexible and extensible. C is not made to do that and you only make a mess when you try to use the tool for something it wasn't made for.


How much nicer is it to have a pure virtual interface to subclass rather than trying to keep up with all the event handler callback functions that carbon demands. As a project grows and evolves, keeping up with all those event handlers and the overhead code that is necessary to register and unregister them becomes a maintenance problem.

Performance is not the only consideration you make when designing a software system. Systems need to be easy to maintain. They need to be easy to extend and they need to be easy for new developers to learn.

-James


-James

October 07, 2006 3:02 PM

 
Anonymous James Gregurich said...

James Gregurich -- the point was that doing it the "Cocoa way", going thru accessors, using the provided collection classes, etc, is slower than doing it the "C way" - direct access to structures, and direct function calls. Yes - maintainability, readability, and robustness may all suffer; for raw speed does come at a price!


How fast do you need a button handler to respond to a mouse click? If it takes a microsecond, are you going to push for a nanosecond?

How fast does a menu need to draw when it is opened?

If you are showing 1 quicktime movie frame per screen refresh, do you need to be faster than that?

There are upper limits to what is needed in the way of performance. Chasing performance just for the sake of chasing performance at the cost of making code less maintainable and extensible is not a good policy.

Correct me if I am wrong, I am not an expert on ObjC, but as I recall, you can obtain a direct pointer to a member function of a class without going through a dynamic lookup. So, if you have a loop where the lookup hit matters, then you can fairly easily bypass it. If one does that, is the procedural call significantly faster? what is it... a difference of two or three instructions to jump indirectly rather than directly?

The best approach is not to pre-optimize code up front looking to get every last ounce of performance. YOu write quality code first, then optimize exactly where it needs optimization.



Consider this....what if your loop has to make calls to different functions depending on some set of circumtances. You would have to put your static calls inside a switch statement or if/else block. That introduces comparisons and branches into the loop body. However, an OO approach would likely use a virtual base class and a set of subclasses to represent the choices. That would remove the branches from the loop. I would suggest the single virtual method call would be much faster than a branch prediction miss or two during the execution of the loop.

now, you could simulate the virtual function call by using a function pointer, but that produces code that is harder to maintain and extend.

I don't accept the assertion that OO code is inherently slower than procedural code. Which one is faster depends on the design of the system and what the system does.

October 07, 2006 3:32 PM

 
Anonymous Stephen Deken said...

So we all agree that FSRefs are useful, if only they were easy to use! Someone else mentioned that it would be neat to have a Cocoa wrapper around FSRef that implemented the NSString methods, so that it could be used in older path-based code.

So that's exactly what I've spent this morning doing. Check out SDFileReference, an immutable subclass of NSString which uses FSRef internally. It can be dropped in to any place you're using NSString to keep a path, and it includes a NSUserDefaults category to read and write the reference.

It's not perfect, but it's a start.

October 07, 2006 4:00 PM

 
Blogger Wil Shipley said...

Johan:

"Uno taco lingua, por favor."

October 07, 2006 4:44 PM

 
Blogger Wil Shipley said...

For one, Quicktime is listed on both pages.

Uh, dude, you have to drill down on the categories to see what is and isn't Carbon. If you click on "QuickTime" from the Cocoa pages you see NSMovie and the newer QTMovie classes, whereas if you do it from the Carbon pages you see all the Carbon APIs I'm complaining about.

October 07, 2006 4:47 PM

 
Blogger Wil Shipley said...

But when you assume that no C API can be good (when that it obviously false, at least to the community) and any proper ObjC interface will be good (again, obviously false) is when you lose respect.

First off:

1) I didn't "assume" anything. I did a bunch of research, and I posted it in a huge posting. If you'd like to post some actual facts that contradict all the actual factual points I made, then we can debate.

2) It doesn't matter what the community thinks. 50% of Americans supposedly also voted for a murderer and a torturer for president. 97% of people use Windows. A million people could think the emperor is five feet tall.

3) I specifically address the issue of bad Objective-C interfaces in my posting (NSMovie vs. QTMovie), so you're just doing a straw man here.

Finally, respect is nice to have, but not at the expense of intellectual integrity. I'd rather have everyone hate me for saying what I feel than turn this blog into a place where I attempt to appease everyone's preconceptions.

Homer: "Please, please, kids, stop fighting. Maybe Lisa's right about America being the land of opportunity, and maybe Adil's got a point about the machinery of capitalism being oiled with the blood of the workers."

October 07, 2006 4:57 PM

 
Blogger Wil Shipley said...

Obj-C speed:

Look, there's no question that sending a message is slower than calling a function. There's no question that accessing a static variable is faster than making a method call.

This isn't the point. You need to measure systemic speed, not the speed of each operation. It's like saying, "Hey, my RISC chip is always going to be faster than your CISC chip, because I can add two numbers in half the time." Well, that didn't work too well for the PowerPC, did it?

You're measuring the efficiency of some common operations and then declaring the speed of the entire system based on them, but in the real world there are two factors that overwhelm anything like native method-call efficiency:

1) You, as a developer, only have so much time to write your app. The less time you have to spend actually fighting the APIs to get the features and glitz you need, the more time you are left to do systemic optimizations (with Shark, yes).

2) Most programs are NOT Adobe Photoshop. That is, most programs don't have tight loops where they perform math operations over and over, and saving a cycle here or there can mean a 10% performance boost.

For most programs, the reason they are slow is because the programmer is doing something REALLY inefficient, like drawing the same thing 10x or, yes, allocating an NSFormatter 100,000 times when he could be doing it once.

This is true of Carbon programs and of Cocoa programs, although in my experience it's less true of Cocoa because, since Cocoa is higher-level, a lot more systemic performance wins have been built into Cocoa for you to get automatically.

For instance, lazy drawing is a feature of Cocoa. Now, in Mac OS X 10.0, there was a lot of duplicated drawing. But in the intervening years, Cocoa engineers have been able to do a lot of optimization behind-the-scenes, and since the NSView drawing code is at such a high level, if you run the same code today on 10.4 as you did under 10.0, it'll be way faster, because it'll do less drawing.

[Remember that it was Cocoa that gave us buffered windows in the first place, and eliminated the ugly white windows that needed their dirty rects filled whenever we revealed them.]

But even when the Cocoa framework isn't natively faster than similar Carbon frameworks, the point is that you, the programmer, will have more time to actually optimize. You'll be able to get rid of all the idiocy and make your code fast.

For instance, when I got image deconvolution working for internal iSights, it could decode a barcode from an image in about a second. I wanted it to take about a hundredth of a second, so it would be responsive AND not burn up the person's CPU when the window was up.

So I refactored my algorithm, and I used more vecLib, and I added some AltiVec / SSE2 directly, and, lo and behold, we got it down to one THOUSANDTH of a second to process an image (on a G4 laptop!).

Method calls simply weren't a factor here. I still have them all over. I didn't make the naive mistake of saying, "Well, this is going to have to be fast, so I better write it all in C!"

When you look at ANYTHING and think, "Is this slow?" you are asking the wrong question.

You need to ask, "Is this slow given how often it is called?"

It's like looking at the car someone drives and labeling it "efficient" or "not efficient". If the owner of a Hummer bikes to work every day, and you commute for an hour in your Prius, well, you're dummy.

That's why I reject any comparison of method calls to function calls -- they don't have any frequency of calling information attached to them.

If you have a routine that takes one hundredth of a second to complete, most programmers would say, "Yah, that's pretty slow". But if it's only called every time a key is pressed -- well, how many keys can the user press a second? I'd wager it's not more than 10.

We've got processors that can do three BILLION operations a second, and you're arguing that Objective-C is slow because it's twelve operations long? What, don't have time in your schedule for four billionths of a second?

Method calls aren't the gating factor for program speed -- good program design is. Cocoa gives you the time to design your program well and then decide how much optimization you want to do afterwards.

Which is how it should be.

-Wil

October 07, 2006 5:17 PM

 
Anonymous Anonymous said...

The problem with zealots is that they never know when they've won.

I think you'll find the Obj-C war has been won. Apple is introducing most of its new APIs in Obj-C and most OSX developers have realized its a better way to put apps together.
The revolution is over man. Its finished.

Yes, there are some old APIs that will need updating but that is happening.
Not as fast as you want, but it is happening.

So either learn to relax man...or.....find another war to fight!

October 07, 2006 9:36 PM

 
Anonymous Anonymous said...

SInce you bring up Photoshop, I'll point out that it's because Photoshop is very low-level code, it's bloody hard to rehost it on top of Core Image (for example).  Adobe had a hell of a time even making it work with 16-bit samples, and who knows if it will ever take advantage of floating point frame buffers?

-jcr

October 07, 2006 9:44 PM

 
Anonymous Bret said...

>> The problem is once you start trying to hack procedural C code when you need it to be dynamic, you make a huge mess that will likely be slower than just using the OO features of an OO language.


Absolutely. Don't Do That! If the day comes where your code that you've designed for pure raw speed needs to suddenly be dynamic, extensible, and generally all those wonderfully OOPish things - chuck it and write a new one in an OOP language. Sometimes you need to trade in the super-bike and get a truck...


>> How much nicer

Er... who said anything about it not being nicer? Of course it's nicer... but in a tight loop it's not faster.

>> How fast do you need a button handler to respond to a mouse click?

Event handling isn't high-load problem space, and hasn't been since... um... well, I could imagine a first gen 8080 machine needing some help.

>> If it takes a microsecond, are you going to push for a nanosecond?

That's a 1000-times increase in speed. Yes - if I get my core data (not to be confused with Apple's Core Data) processing framework (the one that's swamping my machine) to go 1000 times faster, you betcha I would.

>> If you are showing 1 quicktime movie frame per screen refresh, do you need to be faster than that?

But - what if you're not? What if you are asking the machine to process hundreds of frames per refresh? Or what if you are asking the machine to crunch thru a large set of frames as quickly as possible, like when you are rendering a movie to disk (which uses the same framework as playing it to screen)?

Or, what if you aren't the only process in the system? Maybe I want to have a video window open in the corner of my screen, and still be able to use my machine to do something else also? Eating a bunch of CPU arbitrarily is not a good thing... there'd better be a good reason. Oh, and the other thing - eating CPU eats battery life, too...

>> There are upper limits to what is needed in the way of performance.

Depends on the problem space. Event handling is not a major CPU hog; even with all the OOP stuff. Heck - it could even be done OK in (Wil, cover your eyes ;-) Java... However, media processing is, many scientific apps are, 3D apps are, etc...

>> Chasing performance just for the sake of chasing performance at the cost of making code less maintainable and extensible is not a good policy.

Performance is often in itself an enabling factor; so pursuing performance in core frameworks (be they OS core frameworks, or your app's core frameworks) is not a bad goal. I can't do my current project fully OO, it would take to long to run.

>> Correct me if I am wrong, I am not an expert on ObjC, but as I recall, you can obtain a direct pointer to a member function of a class without going through a dynamic lookup.

No. But, once you have gone thru the lookup, you can cache that and call it directly. But, the result is a bit messy, also...

>> a difference of two or three instructions to jump indirectly rather than directly?

In a tight loop running over a huge dataset, that matters. BTW, IIRC, in some cases the cached call can be quicker, as there's slightly less stack setup and tear-down involved (um... I thought I read that either over at ridiculousfish.com or www.mulle-kybernetik.com???)

Now, doing this breaks down if there's any chance of polymorphism in the set that you are iterating over, as all of a sudden you've got the wrong method implementation pointer... which is something somebody else might well assume was a valid thing to throw at you ("Huh? wadda' you mean I can't subclass that? why do you think I learned OOP - so I can derive everything from the top object?!?"). That can't happen with a C framework, because there's no notion of subclassing (not that that's a good reason to avoid OOP - just that it's a potential pitfall that you will run into when trying to circumvent the runtime). Note that toll-free-bridging also has this pitfall.

Now, I'm not advocating forgoing OOP without good reason - one should have a good idea of what the framework in question is gonna' look like, and where the bottlenecks are (otherwise, it's just another case of premature optimization).

>> now, you could simulate the virtual function call by using a function pointer, but that produces code that is harder to maintain and extend

Depending on exactly how you implement it, that's not a simulation, it's exactly what the compiler is generating. A typical OOP implementation will have a VMT pointer at the front of the object (obj-C calls it an isa, and it actually points to the class object, at which points it starts getting a little complex what with selectors and caching and all) and a table of pointers to method implementations somewhere else - so, 2 de-references to call through if you are in something traditional like C++. If you implemented it your self you could put the function pointer right in the object, and reduce that to one de-reference.

But, if you find yourself in this pickle, I might suggest trying to refactor the problem such that that branching can be avoided in performance-critical areas...

>> I don't accept the assertion that OO code is inherently slower than procedural code. Which one is faster depends on the design of the system and what the system does.

It's faster because it's an apples-to-oranges comparison: fully "OOP-ified" code (i.e., going all the way with abstraction, etc) inherently does more than fully "procedural-zed" (i.e., taking every specific-case optimization you can) code does. All else equal, doing less is always faster than doing more.

That said, you can take my Cocoa away when you pry it out of my cold, dead fingers!


Wil - I presume you are referring to my comments - so...

>> 1) You, as a developer, only have so much time to write your app.

Developer efficiency != raw code efficiency. Actually, they tend to follow an inverse relationship... (but, fortunately, there also tend to be "hot-spots" in one's code where one can concentrate one's efforts for maximum bang-per-buck). I wasn't talking about developer efficiency, just pure down-to-the-metal raw speed, which is (and was much more so in the past) important to a number of "Carbon" frameworks, like QuickTime.

>> 2) Most programs are NOT Adobe Photoshop.

Nope! And thank goodness, too! I've got the source for 5 different non-trivial apps of mine sitting in my development folder, and only 1 of them has any significant non-OOP (or even non-Cocoa) content... But - some frameworks (again, like QuickTime, or some scientific-type stuff like what I'm working on now) are sensitive like Photoshop, and shaving a cycle here and there does make a difference.

>> For most programs, the reason they are slow is because the programmer is doing something REALLY inefficient

Er... of course. See above bit about most programs not being Photoshop... so it only stands to reason that if you aren't doing much, and are still doing it slowly, well, somethin' gotta be busted! :-)

>> a lot more systemic performance wins have been built into Cocoa for you to get automatically.

Yup. And continue to get as the underlying framework gets improved upon. But, if you do everything with a high-level procedural API, that gets systemic improvements, you will also get that win, too (not sure why you'd want to actually have a high-level procedural API, as almost all high-level problem spaces lend themselves to OOP much better than they do to procedural programming, and high-level operations tend not to involve things that require super-intensive-hyper-efficency, but I digress...)

>> the point is that you, the programmer, will have more time to actually optimize.

That's a people problem, not a machine problem. (now, granted, it's also true, and we are, after all, people - so, yeah, at the end of the day it really makes much more sense for stuff like DL or OmniOutliner or whatnot to be done in 100% Cocoa; I'm not now nor was I ever disputing that).

>> Method calls simply weren't a factor here.

That's because your hotspot was in a single function (or perhaps a few - I haven't seen your code). And function calls vs inlining wouldn't have been a factor either, as you most likely weren't crossing function boundaries in your hotspot. Large, diffuse hotspots don't have that luxury, and in some cases, it can make a real difference - but, you need to know what to look for (so, model the thing in advance to determine what needs that level of attention).

>> If you have a routine that takes one hundredth of a second to complete, most programmers would say, "Yah, that's pretty slow". But if it's only called every time a key is pressed -- well, how many keys can the user press a second? I'd wager it's not more than 10.

So, now you are taking up 10% of the CPU because I'm typing a note (or just chatting away) in the forground while I'm waiting for some operation to finish in the background? That doesn't seem very nice...

>> We've got processors that can do three BILLION operations a second,

More - you forgot that they're superscalar. 8-)

>> and you're arguing that Objective-C is slow because it's twelve operations long?

Yup. 12 times slower. Needing - or not needing - that 12 times speedup depends on context.

>> What, don't have time in your schedule for four billionths of a second?

Not when I'm repeating that so often that I need scientific notation just to count it all... (remember - some things are Photoshop!)

>> Method calls aren't the gating factor

Not the factor, just a factor in some cases. The wrong algorithim, optimised to the hilt, will allways be slower than then the right one in it's nave state... but, the right algorithim, fully optimised, will blow both of them away.


BTW - something people seem to be missing - when dealing with a hard problem space, where you know (or even just have a hunch) that you are gonna' need every ounce of performance you can get and then some, it really pays to model it ahead of time (or in other words, build one to throw away). Then, when you sit down to write the "final" version, you will already have a good understanding of the problem space, and what your product is gonna' look like. Consequenly, the decision to use C (or not to use C, or do anything else for that matter) for any particullar part won't be made in a vacuum of knowlege, and won't be a premature optimization attempt.


P.s. - to anybody who thinks that I'm some sort of OOP-hate'n luddite or procedural zelot - let me assure you that I'm not; I love OOP (and specifically, Cocoa) a lot, it's just that I recognise the cost that it entails, as well as the (immense) benifits.

And yes, Wil - you may well be crazy, but QuickTime is obtuse! ;-)

October 07, 2006 9:49 PM

 
Anonymous James Gregurich said...

bret,

If you have a loop that does so little real work per iteration that the virtual function call overhead matters, then what could that loop possibly be doing that is useful in the real world?

If you are doing a physics simulation, you are going to be concerned far more with how much time a square root calculation takes than the couple extra cycles a virtual function call is going to take.


ofcourse you make the code as fast as it needs to be. My point is that the situations where the use of virtual function calls is going to be the cause of a performance problem are very, very rare. Religious use of purely procedural code here isn't going to buy you any significant performance gain.

In the context of the carbon APIs, you are already having to deal with indirect jumps to callback functions. Core Video (for instance) calls a callback function for each frame. What is the performance difference between that and calling a virtual function on a supplied pure virtual interface?

Because the designer chose to make the callback a flat function instead of a part of a OO virtual interface, I now have the extra hassle of having to manually deal with the registration of the callback.

You might as well design a proper OO API and make the life of the developer easier so that he can ship his software more quickly.

-James

October 07, 2006 11:15 PM

 
Anonymous James Gregurich said...

Yup. 12 times slower. Needing - or not needing - that 12 times speedup depends on context.

that is not a correct analysis.

if a function consists of 1000 operations, adding an additional 12 operations is a 1.2% difference. That generally isn't a significant hit.

Take a look at how many instructions are used for a simple function call. There are what...15 or 20 extra operations per function call to manage the stack, store and restore registers and what-not.

If your claim is substantial in practice, then one shouldn't even be using procedural function calls due to the added overhead. Where does one stop on trying to squeeze out every last cycle?

-James

October 07, 2006 11:27 PM

 
Blogger Drew Thaler said...

Cannot the authors of the Finder and Itunes put C or C++ wrappers around a ObjC API?

Nope. For all that ObjC is runtime dispatched, it still requires some static linking of C symbols -- eg, "objc_msgSend" and "objc_class_name_NSObject" -- and proper initialization of the runtime. If you were going to start putting ObjC into system frameworks used by non-ObjC apps, those apps would have to become ObjC apps by linking against libobjc and Foundation.

We could keep going along the lines of "but couldn't Apple force that?" or "but couldn't libobjc get merged into libSystem" or "but couldn't everyone just re-link their code?" ... but the short answer to all of them is simply no. For various and sundry really good technical reasons, no. :-)

I see no reason why that can't be dealt with by the appropriate use of subclassing, namespaces, and versioned frameworks. Perhaps someone can educate me

Read the Wikipedia link I gave so that you understand fragile binary interfaces and fragile base classes. Got it? Okay. Subclassing and namespaces do absolutely nothing to help that. So let's look at versioned frameworks.

Almost every system update Apple has ever released over five years involves a patch to at least one library or framework, often one that means adding APIs to deal with some new feature or other and which thus would break binary compatibility in C++.

How many framework versions of every framework do you think you would need to ensure that an app written for 10.0 and every other version would still run in 10.4? In other words, how many times has each framework has been revved since 10.0? Easily 10 or 20 times, and probably triple digits for some. Now look at the sizes of your /S/L/Frameworks, /S/L/PrivateFrameworks, and /usr/**/lib directories ... and multiply by that number to get a ballpark of your new OS disk space requirements. That 80GB hard drive isn't looking so big now, is it?

Bonus question #1: Compute the amount of engineering and testing resources that it would take to fix and test a bug that exists in (a) a framework, (b) 20 versions of a framework.

Bonus question #2 (rhetorical): Starting with Apple's current cash on hand figures, compute how long the company could afford to burn money at the rate described in question #1b prior to bankruptcy.

Bonus question #3: Why is the argument that "but you could still manage to do it if everyone who ever wrote software was a really good programmer and everyone was careful and cooperated nicely" not a useful contribution? :-)

October 08, 2006 12:21 AM

 
Blogger Drew Thaler said...

There is also an important point that some of the younger people here seem to be completely forgetting.

Back in 1997-2001, if Apple had not been worried about compatibility with existing bodies of C and C++ code -- by which I mean, literally, ALL the Mac code that ever existed at the time -- it would have wound up with developers and users completely abandoning it. It would've been NeXTStep 2006, by which I mean a tiny niche player that most of you would never have heard of. Not Mac OS X.

Some of you are arguing that NeXTStep 2006 would've been better. With due respect, you are wrong. :-) A little bit of chaos and compromise goes a long way toward making the system robust. Ten years from now, if Scheme or Dylan or something I dont even know about becomes the dominant paradigm and Cocoa has died off, OSX will interface with it seamlessly.

October 08, 2006 12:23 AM

 
Anonymous Anonymous said...

Some of us would be happy if Apple just fixed all the bugs they have in the existing APIs....

Sheesh!

October 08, 2006 1:47 AM

 
Anonymous bret said...

OK. Now - this is only for the tight-loop case; and only when you are really cpu-bound, and this example is from a section of my current project (which is a bit odd). And this is a bit of an approximation, just off the top of my head. But...

The "Cocoa Way", observing encapsulation and abstraction: Per inner loop iteration, I come up with 4 objective-C dispatches, a floating point multiply, a floating point add, a move, an integer test, and a conditional jump. On top of that, there's the overhead from using a NSEnumerator (obtained from a NSSet). There may be some extra moves thrown in there in Intel, too. So... say, 60 or 70 ops, depending on what exacly is going on in the accessors?

The "Procedural Way", which doesn't have any notion of encapsulation and abstraction, and just goes streight to the structures: I come up with a pointer move, a floating point multiply, a floating point add, an integer test, and a conditional jump. There may be some extra ops involved on PPC, because (IIRC) Intel can do a pointer operation in the same operation with a data operation (i.e., work directly in memory pointed to by some register plus an offset that's attached to the instruction). So... maby 5 or 10 ops? 20 tops?

Note that this particullar case that I've got can't be vectorised easally because the doggon graph keeps changing as you're running thru it (that's the outer loop's doings) - you'll spend as much time setting up for the vector engine (or worse) as you will just doing the main op direcly on the graph.

Oh, and BTW, the sqrts all happen in the outer loops, where it's not so much of a hit...


:-)

October 08, 2006 2:29 AM

 
Anonymous Anonymous said...

Bret,

I hit a brick wall in trying to find the setting for accessing the focus attribute

The code to set the focus is inside the following sourceforge project: osxrecognition it must be what the direct competion uses.

Here is the method for your convience to copy paste – good luck with your project:

- (void)setFocus:(float)aFocusValue onChannel:(SGChannel)sgchanVideo {
// Focus to (.1)
long version;
OSErr result = Gestalt(gestaltQuickTime, &version);
if ((result == noErr) && (version >= 0x06408000) && aFocusValue >= 0 && aFocusValue <= 1) { //check for 6.4 quicktime
QTAtomContainer iidcFeaturesAtomContainer = NULL;
QTNewAtomContainer(&iidcFeaturesAtomContainer);

QTAtom featureAtom = nil;
QTInsertChild(iidcFeaturesAtomContainer, kParentAtomIsContainer, vdIIDCAtomTypeFeature, 1, 0, 0, NULL, &featureAtom);
QTAtom typeAndIDAtom = nil;
VDIIDCFeatureAtomTypeAndID featureAtomTypeAndID = {vdIIDCFeatureFocus, vdIIDCGroupMechanics, {5}, vdIIDCAtomTypeFeatureSettings, vdIIDCAtomIDFeatureSettings};
QTInsertChild(iidcFeaturesAtomContainer, featureAtom, vdIIDCAtomTypeFeatureAtomTypeAndID, vdIIDCAtomIDFeatureAtomTypeAndID, 0, sizeof(featureAtomTypeAndID), &featureAtomTypeAndID, &typeAndIDAtom);
QTAtom featureSettingsAtom = nil;
VDIIDCFeatureSettings featureSettings = {{0, 0, 0, 0.0, 0.0}, {vdIIDCFeatureFlagOn | vdIIDCFeatureFlagManual | vdIIDCFeatureFlagRawControl, aFocusValue}};
QTInsertChild(iidcFeaturesAtomContainer, featureAtom, vdIIDCAtomTypeFeatureSettings, vdIIDCAtomIDFeatureSettings, 0, sizeof(featureSettings), &featureSettings, &featureSettingsAtom);

VDIIDCSetFeatures(SGGetVideoDigitizerComponent(sgchanVideo), iidcFeaturesAtomContainer);
//You can pass the ComponentInstance to GetComponentInfo() and it will fill out a ComponentDescription that includes the subtype.
}
}

October 08, 2006 2:30 AM

 
Anonymous Anonymous said...

i'm pretty sure Steve Jobs would have scrapped Carbon altogether if it had been possible. But, as indicated already above, it was kind of a business necessity that stood against such a clear cut (but see http://rixstep.com/1/1/20061006,00.shtml), the transition had to be kept smooth in order to guarantee return on investment on part of third-party developers). And we also should keep in mind that some amount of voodoo was necessary to bridge that giant gap, "making a C API that knows about Objective-C objects"

http://ridiculousfish.com/blog/archives/2006/09/09/bridge/

"OS X is a nifty hard core UNIX with the best GUI ever built on top." (... and soon with DTrace under the hood)
http://soiland.no/doc/osx

October 08, 2006 3:19 AM

 
Blogger Wil Shipley said...

osxrecognition it must be what the direct competion uses.

I hope not. That looks an AWFUL lot like my copyrighted, unpublished code. I don't necessarily want to throw a bomb here, but, uh, pretty much EVERY variable name is exactly the same as my code, my comment is duplicated almost exactly, the if statement at the beginning is the same even though it's bad coding in that context (why wouldn't you check the contrapositive and just return early in this case if you don't have QT 6.4?), and every statement is in the same order as my code, and the variables are all initialized just-in-time as in my code, and they make the same mistakes as were present in the 1.0 version of my code -- QTAtoms are initialized as if they are pointers in both code samples (nil is an empty pointer to an object, NULL is an empty void *), when in fact QTAtoms are typedefed from "long", but back when I wrote version 1.0 the compiler didn't care if you switched NULL for 0L. In the last two years Apple added a check and doing this now emits a warning, but using 'nil' for 0L still does not emit a warning.

Here's my code from 3/28/2004, pulled from my subversion database:

long version;
err = Gestalt(gestaltQuickTime, &version); // check the version of QuickTime installed
if (version >= 0x06408000) { // we have version 6.4 -- so we can set autofocus off!
ComponentDescription componentDescription;
err = GetComponentInfo((Component)videoDigitizerComponent, &componentDescription, NULL, NULL, NULL);
NoteErr(err);
if (componentDescription.componentSubType == vdSubtypeIIDC) {

// SET the focus to be manual
QTAtomContainer iidcFeaturesAtomContainer = NULL;
QTNewAtomContainer(&iidcFeaturesAtomContainer);

// 'feat'
QTAtom featureAtom = NULL;
err = QTInsertChild(iidcFeaturesAtomContainer, kParentAtomIsContainer, vdIIDCAtomTypeFeature, 1, 0, 0, NULL, &featureAtom);
NoteErr(err);
// 't&id'
QTAtom typeAndIDAtom = NULL;
VDIIDCFeatureAtomTypeAndID featureAtomTypeAndID = {vdIIDCFeatureFocus, vdIIDCGroupMechanics, {5, 'F', 'o', 'c', 'u', 's'}, vdIIDCAtomTypeFeatureSettings, vdIIDCAtomIDFeatureSettings};
err = QTInsertChild(iidcFeaturesAtomContainer, featureAtom, vdIIDCAtomTypeFeatureAtomTypeAndID, vdIIDCAtomIDFeatureAtomTypeAndID, 0, sizeof(featureAtomTypeAndID), &featureAtomTypeAndID, &typeAndIDAtom);
NoteErr(err);
// 'fstg'
QTAtom featureSettingsAtom = NULL;
VDIIDCFeatureSettings featureSettings = {{0, 0, 0, 0.0, 0.0}, {vdIIDCFeatureFlagOn | vdIIDCFeatureFlagManual | vdIIDCFeatureFlagRawControl), 0.60}};

err = QTInsertChild(iidcFeaturesAtomContainer, featureAtom, vdIIDCAtomTypeFeatureSettings, vdIIDCAtomIDFeatureSettings, 0, sizeof(featureSettings), &featureSettings, &featureSettingsAtom);
NoteErr(err);

err = VDIIDCSetFeatures(videoDigitizerComponent, iidcFeaturesAtomContainer);
NoteErr(err);

QTDisposeAtomContainer(iidcFeaturesAtomContainer);

err = SGVideoDigitizerChanged(sequenceGrabberChannel);
NoteErr(err);
}
}

Yah, that COULD be wild coincidence, that this person happened to use all the same variable names as me. Maybe there's someone out there who has independently developed the exact same variable-naming scheme as me, even though I've never met anyone who has it?

I'd very much like to talk to the person who posted this on SourceForge. (Oddly, the code you posted looks a LOT more like my code than the version that's on SourceForge right now -- did you change this, Mr. Anonymous?)

October 08, 2006 6:25 AM

 
Anonymous Anonymous said...

A company needs to consider their consumers. In the developer community, you have both procedural programmers and OO programmers and those who can jump in and out of both. Apple has to satisfy their consumers. As it happens ,in real life, it takes a lot longer for transitions to happen. Sometimes an entire generation must go by before old ways are set aside. And, with Apple wanting Windows developers to develop for the Mac, Procedural APIs are still important to them. What Apple needs to do with their APIs is make sure they are available on their target platforms, which includes Windows. If Cocoa is going to completely replace Carbon, than Cocoa must work on the Windows platform and Apple's developers tools must be able to generate Windows Apps.

October 08, 2006 9:17 AM

 
Blogger Alberto said...

The Carbon choice/gamble was wrong. And it perpetuation is just silly. To add another disgraceful api take a look at the DirectoryServices.framework apis. These are wose than the others not only because they are verbose like their LDAP cousin (yuck), but the syntax of function names and other things don't match the captitalization of things like Carbon. I have no way of describing it with the exception that this engineer didn't write anything apple in Cocoa or Carbon.

October 08, 2006 11:02 AM

 
Anonymous James Gregurich said...

If you were going to start putting ObjC into system frameworks used by non-ObjC apps, those apps would have to become ObjC apps by linking against libobjc and Foundation.

why is that a problem? most OSX apps use the Objc runtime. Why is it a sin if the FInder does too?

Read the Wikipedia link I gave so that you understand fragile binary interfaces and fragile base classes. Got it? Okay. Subclassing and namespaces do absolutely nothing to help that.

ok. so lets get into the practical specifics about this.

You publish library libDiskBurning.dylib that contains a class iBurnerDevice.

iBurnerDevice has a set static factory method( deviceByWhatever(...); ) that allows one to query private iBurnerDevice subclasses. The client works with the abstract interface to work with the device. This constitutes version 1. Put these class and related structures in namespace nspDiskBurning_v1

a year goes by, new devices come out that requires new functions.

create nspDiskBurning_v2. in this namespace, subclass:


class iBurnerDevice : public nspDiskBurning_v1:: iBurnerDevice
{
...
};


if necessary, the version 2 device class can offer a new deviceByWhatever() factory method to return the version 2 burner device class instances.

ok, in this scenario, old code will continue to link against the version 1 classes which are in their own namespace and new code has the option of using either version 1 or version 2.

ok. I don't see anything here that breaks old code when version 2 comes out. You don't even need to have a new incompatible version of the lib as the old stuff remains unchanged and new stuff is all segregated into its own sandbox. Nothing here relies on class sizes changing as the disk burner class instances are returned by reference.

So tell me, where in this example or there any maintenance or extensibility issues? I'd argue that the approach described is incredibly extensible...much more so than a plain procedural C API. Its also alot more convenient and flexible for the application developer.

I'd also like to point out that is is basically how you currently talk to the IOKit from a user space application. you call a function to request a set interface (structure of function pointers) that are built in the kernel by the upper layers of the device driver stack. It uses a rudimentary form of COM.

-James

October 08, 2006 1:33 PM

 
Anonymous Bret said...

Thanks for the pointer about the focus... whoever you are... :-)

Wil - um... the variable names don't look that unusual to me (actually, it has similar flavor to much of my code), and I suspect there's only a few sequences of operations that will yield working results. Your comment about that if statement - while correct that it's poor practice - misses that that poor practice is actually rather common.

So, just on the basis of that particullar fragment - I'm not so sure that you do have a security leak. Course' I (obviously) haven't compared the rest of that project to your codebase, though, so... ???

October 08, 2006 1:43 PM

 
Anonymous James Gregurich said...

Back in 1997-2001, if Apple had not been worried about compatibility with existing bodies of C and C++ code

I certainly recognize the necessity of the Carbon APIs. I'm sure Wil does as well.

However, our basic point is that those days are over and OSX's new APIs need reflect modern software design principles. The primary API on OSX needs to be cocoa with carbon playing only a legacy role. The major components of OSX should have modern OO APIs instead of poorly documented, overly complicated procedural APIs.

There is no law that requires "low level" APIs to be procedural C APIs. ObjC classes can be called from C++. ObjC classes can be wrapped to be interfaced with with procedural C code.

Whatever system comes in the future can just as easily be mapped back to cocoa classes as it can to procedural C APIs.

I'm not sure who you consider to be "younger", but I have been using Apple platforms since the Apple IIe. I was here for the dark days of the mid 90s.



-James

October 08, 2006 1:45 PM

 
Anonymous James Gregurich said...

So... say, 60 or 70 ops, depending on what exacly is going on in the accessors?

ok. add a function to your contained class that does all the calculations and returns a result so that on each iteration, you have one function call. Inside, that function, you have direct access to your data members.

Secondly, cache the function call ahead of the loop and reuse it for each iteration.

How much does that reduce your operation count?

If you can do that, you've reduced your operation count and maintained your data encapsulation.

October 08, 2006 1:59 PM

 
Anonymous James Gregurich said...

And, with Apple wanting Windows developers to develop for the Mac, Procedural APIs are still important to them.

.NET is not procedural. Microsoft development is moving toward object orientation as well. Win32 is legacy.

-James Gregurich

October 08, 2006 2:03 PM

 
Anonymous Anonymous said...

Wil Shipley:Ok, so a couple people have asked on this list how to actually set parameters on IIDC cameras

You were kind enough to share your knowledge with the QuickTime list, so that is probably were the posted derivation of the method came from.

What would be interesting is if you would share your new version so that Bret, others and I can learn the new and improved Wil way for our own projects.

Carbon is tough for Cocoa programmers, I wonder if Cocoa is as tough for Carbon programmers?

October 08, 2006 2:06 PM

 
Anonymous Anonymous said...

"Yah, that COULD be wild coincidence, that this person happened to use all the same variable names as me. Maybe there's someone out there who has independently developed the exact same variable-naming scheme as me, even though I've never met anyone who has it?"

I think you may be jumping to conclusions. These are the suspicious variable names:

QTAtomContainer iidcFeaturesAtomContainer;
QTAtom featureAtom;
QTAtom typeAndIDAtom;
VDIIDCFeatureAtomTypeAndID featureAtomTypeAndID;
QTAtom featureSettingsAtom;
VDIIDCFeatureSettings featureSettings;

Hardly anything unique. I've never seen this code before, but it's likely I might have named these variables the exact same way.

October 08, 2006 2:40 PM

 
Anonymous Andrew Welch said...

Seriously, Objective-C good, C bad. Cocoa has advantages like having a dynamic object model, being able to subclass system classes, having somewhat decent memory management, and even uncomfirmed rumors of garbage collection coming to Objective-C.

Carbon doesn't cut it. It's bad. Bad bad bad. There are ways to write better Carbon, but there aren't reasons to do so. It should go away.


It's really difficult to take you seriously if that's your view.

October 08, 2006 2:42 PM

 
Blogger Pilky said...

I'm sorry, but that quote kinda shows you up Wil. It makes you seem like a Cocoa troll. Also, I'm pretty sure it's common knowledge that GC is in Obj-C:

"Objective-C 2.0

So compelling, Apple wrote Xcode 3.0 itself using it. Enjoy modern garbage collection, syntax enhancements, runtime performance improvements, and 64-bit support."

http://www.apple.com/macosx/leopard/xcode.html

I agree that it would be nice to have Cocoa wrappers for a lot of frameworks, I don't like Carbon much either. But the amount of people that flat out refuse to use Carbon is astounding. As much as you may dislike something if you have to use it, you have to use it. I personally hate NSURL, but I have to use it sometimes. People are given the tools they need to make applications, whether they choose to use them is another matter. To me saying you just want to use Cocoa so you don't have to use Carbon is like saying you want to just use a screwdriver so you don't have to use a hammer. I'd much rather Apple fix the bugs in the screwdriver rather than trying to add a hammerhead to it

October 08, 2006 3:58 PM

 
Blogger Wil Shipley said...

On the IIDC code: Oh! That explains it! I shouldn't post at 4am! I did a google search to see if I'd posted it before, but didn't find anything... shoulda waited until morning.

Andrew: I am sorry you are unable to take me seriously, but I also strongly feel that I have to be me.

October 08, 2006 4:18 PM

 
Blogger Wil Shipley said...

Pilky: Your arguments honestly make no sense. Do I have to use tools I dislike, or not? You say both within two sentences.

Carbon isn't a screwdriver, it's a packet of ramen.

October 08, 2006 4:25 PM

 
Blogger Wil Shipley said...

PS: I'm glad you found an official source for the garbage collection thing so I don't have to quote third parties and call it a rumor.

October 08, 2006 4:26 PM

 
Anonymous Lon D. Varscsak said...

Mmmmm....ramen. ;)

October 08, 2006 7:13 PM

 
Anonymous Bret said...

James -

No can do. The bits of the calculation are spread over several structures (and cannot be readily re-factored). Also, ideally you're supposed to always access ivars thru accessor methods, even within the same class - so directly accessing them in a single calc method is - from a pure OOP standpoint - "cheating" (not that I haven't been known to "cheat" to get the job done from time to time myself :-)

Oh, and speaking of "cheating" - imp caching is also cheating, because it breaks subclassing in the dataset; and - unless you are careful with your implementation - so is toll-free-bridging for the same reason.


Wil - No, it's not. Ramen is quick to prepare, convenient, and reasonably tasty. And cheap. Really cheap!

Carbon is none of those things... No, Carbon is more like a 3-day-old brussle-sprout casserole, that got somewhat burnt - not too tasty to begin with, comparatively expensive and involved to put together, and now somewhat of a mess, but still full of essential vitamins (like BC and LIO - that's vitamin Backwards Compatibility and Legacy InterOperability for those of you not up on your OS nutritional charts ;-) even though it makes you grimace as you eat it.

Which is not to imply that if your particular body chemistry doesn't need BC and LIO, that you should be forced to eat it...

:-)

October 08, 2006 9:40 PM

 
Anonymous Rosyna said...

Bret, I'm really not sure I want to know how you use screwdrivers...

October 08, 2006 9:58 PM

 
Anonymous Stephen Deken said...

The wrapper around FSRef that I mentioned yesterday has been updated to address performance and correctness concerns:

SDFileReference

Still not perfect, but better than yesterday. BSD license.

October 08, 2006 11:42 PM

 
Blogger Drew Thaler said...

So tell me, where in this example or there any maintenance or extensibility issues?

Nowhere. But you've completely failed to come up with a useful C++ API.

In your example you are requiring that the client call exported factory methods to create objects. That makes it impossible for the client to subclass an object, and therefore completely destroys the OOP benefits of the code.

And that was the whole point, remember?

We're not just doing this so that we can change the syntax of our calls to "device->getProperties()" instead of "DRDeviceGetProperties(device)". The goal is to actually have a useful OOP paradigm.

Without subclassing you've created a design with pretty much the worst of both worlds -- C++ syntax, but C behavior. You've now got mangled symbols, multiple obsolete namespaces, and a growing inheritance tree ... but clients still can't override a virtual function.

By the way, in case it wasn't clear to you, the underlying implementation is generally written in C++ for most things. It's just glued back to a C API to be exported because C APIs lack the fragility problems.

October 08, 2006 11:45 PM

 
Blogger Pilky said...

Pilky: Your arguments honestly make no sense. Do I have to use tools I dislike, or not? You say both within two sentences.

Carbon isn't a screwdriver, it's a packet of ramen.


Sorry, I missed a comma. It should have said:

"As much as you may dislike something, if you have to use it, you have to use it."

Pretty much what I'm saying is that the people who refuse to even consider carbon aren't very good programmers. I personally won't write carbon, I really don't like it, I find writing in most languages other than Obj-C horrible. However if find a solution to my problems that involves using some carbon code people have supplied then I'll use it.

As for the screwdriver analogy, Cocoa is the screwdriver, Carbon is the hammer and the people who refuse to use Carbon want Apple to add a hammerhead to the screwdriver than making a better screwdriver. There are lots and lots of bugs or annoying limitations with Cocoa for which Carbon has no answer, why not focus on fixing those rather than trying to add something just so a few people don't have to venture outside of Cocoa. The only two major Carbon frameworks I could think of that lots of people might use are Quicktime and Keychain, both of which now have Obj-C wrappers.

October 09, 2006 3:01 AM

 
Anonymous Benji XVI said...

what I'm saying is that the people who refuse to even consider carbon aren't very good programmers
Then your are saying nothing about whether Carbon should stick around.

Wil's point is that Carbon makes things unnecessarily difficult where an Obj C 'wrapper' could make them easy as pie (mmm). He's talking about the way things could be - and should be.

By saying that these difficult-should-be-easy things are still possible with Carbon you add nothing to the debate and merely re-write that which Wil made abundantly clear in his article.

October 09, 2006 6:28 AM

 
Blogger Pilky said...

I'm just saying that I'd prefer Apple to fix many of the bugs or annoyances in Cocoa as it is now, instead of wasting time adding wrappers to APIs that are already there. As Tiger showed, Apple will provide wrappers to frameworks if there's enough need for them eg QTKit.

Imagine you were writing a website and you just new PHP but you wanted fancy client side effects. You'd just use Javascript, you wouldn't start complaining about how you're having to use Javascript and that browsers should implement client side PHP.

All I'm saying is that the programmers that refuse to use both frameworks no matter what are just as bad as the people that say that their platform is better no matter what. Those who have been acting like Carbon is useless and Cocoa is god are just like those that would go as far as to recommend a Mac over a PC for a gamer ie. too biased to give a decent opinion.

As I have pointed out, I don't like Carbon at all, I think it's horrible to work with compared to Cocoa. Though I've been brought up on Cocoa. If I had been brought up on Carbon I might feel the same about Cocoa. The thing is, I acknowledge that Carbon is better than Cocoa at some things. Why don't those that dislike Carbon so much and would rather use Cocoa write Cocoa wrappers for those Carbon functions?

October 09, 2006 8:39 AM

 
Anonymous Robert van Uitert said...

Wil, have you ever looked at HIViews? They do use modern, OO paradigms. And their use of Carbon events is strikingly similar to how ObjC handles messages. Most of "modern Carbon" uses modern paradigms and any new Carbon app uses mostly modern Carbon, not the Carbon that was used to carbonize OS 9 apps. (Modern Carbon won't even run on OS 9, so it's not a compatibility layer anymore.)

Some people just don't like to use ObjC and some of them even have pretty good reasons for that. Personally, I just don't like the ObjC syntax. I think Cocoa is great, but the fact that I prefer C++ makes me use Carbon more than Cocoa. My app (FileBrowse) does use Cocoa, but mostly where the functionality is missing in Carbon or Core Graphics, etc.

I admire you for what you accomplished with Delicious Library, and ObjC/Cocoa certainly works great for you, but it's not for everyone. It took me about a year to finish version 1 of my app, but I spent more time designing than coding, so let's say I coded for 6 months. Maybe that would've been less if I had used Cocoa all the way, I will not deny that, but I would not have enjoyed working on my app so much then. That's just a personal thing; we all enjoy different things.

By the way, if you say QuickTime and Core Audio are Carbon, then shouldn't Core Graphics be called Carbon also? CG you do seem to like, kind of. CG, HIViews, and Carbon events all look very similar, using the same "modern Carbon" conventions.

October 09, 2006 1:01 PM

 
Anonymous Bret said...

Rosyna - I use a screwdriver like anybody else; mostly as a chisel or a prybar, and sometimes to mix up my metaphores. ;-)


Pilky - one reason many folks don't write wrappers (or at least publish them) is that then they are the one's stuck with dealing with maintaining the Carbon interface for everybody else...

October 09, 2006 1:05 PM

 
Anonymous Lon D. Varscsak said...

I drink my screwdrivers.

October 09, 2006 2:28 PM

 
Anonymous Ben said...

[Wil Shipley]
All: I absolutely agree there should be an object-oriented file class to replace paths. Handling paths isn't fun.

I just think FSRef isn't it. In part because it's object-oriented, and part because I find it hard to use since, through no fault of its own, it wasn't really designed for long paths, Unicode paths, and UNIX filesystem conventions.

I certainly understand that there would be some loyalists to FSRef while we wait for the Foundation team to give us an NSFileReference or some such.


According to the NSURL Class Reference:
"NSURL objects can be used to refer to files, and are the preferred way to do so."

NSURL is used by NSDocumentController and NSDocument, and is also the chosen Cocoa equivalent to AppleScript's "file" type.

So, instead of designing a new class (such as NSFileReference), would it be useful to add FSRef support to NSURL?

Or, since NSURL is "toll-free bridged" with CFURL, you could just use the CFURLCreateFromFSRef and CFURLGetFSRef functions.

October 09, 2006 5:04 PM

 
Anonymous Rosyna said...

ben, that is incorrect. NSURL is definitely not the preferred way to refer to files. And NSDocument uses an FSRef internally to refer to files.

CFURLRefs were only the preferred way to handle AppleScript/AppleEvents because FSRefs weren't valid across processes. In 10.4, they're valid across processes so there's no reason not to use them for AppleEvents any more (remember, AppleEvents support automatic coercion to other types if needed, so passing an FSRef will still work)

October 09, 2006 6:07 PM

 
Anonymous Rosyna said...

ben, that is incorrect. NSURL is definitely not the preferred way to refer to files. And NSDocument uses an FSRef internally to refer to files.

CFURLRefs were only the preferred way to handle AppleScript/AppleEvents because FSRefs weren't valid across processes. In 10.4, they're valid across processes so there's no reason not to use them for AppleEvents any more (remember, AppleEvents support automatic coercion to other types if needed, so passing an FSRef will still work)

October 09, 2006 11:19 PM

 
Anonymous Anonymous said...

@drew thaller

various and sundry = redundant

October 10, 2006 10:06 AM

 
Anonymous Ben said...

[Rosyna]
ben, that is incorrect. NSURL is definitely not the preferred way to refer to files. And NSDocument uses an FSRef internally to refer to files.

CFURLRefs were only the preferred way to handle AppleScript/AppleEvents because FSRefs weren't valid across processes. In 10.4, they're valid across processes so there's no reason not to use them for AppleEvents any more (remember, AppleEvents support automatic coercion to other types if needed, so passing an FSRef will still work)


I'm not a Cocoa expert; I was just quoting from Apple's documentation: NSURL Class Reference and Cocoa Scripting Guide.

But I've noticed that the most recent classes use NSURLs, for example see the PDFDocument Class Reference.

Does FSRef support the movement of open documents, or would an AliasHandle be required?

Maybe the Foundation team could design a private subclass of NSURL, with an AliasHandle instance variable. It could call FSResolveAlias or FSResolveAliasWithMountFlags whenever the path or URL is accessed.

October 10, 2006 10:53 AM

 
Blogger Drew Thaler said...

@Anonymous

"Various and sundry" is not redundant. Two different shades of meaning are involved: "various" emphasizes the multiplicity, while "sundry" emphasizes the diversity. It's also a phrase that's been in use for centuries and is still popular today, even though it's reached cliché status.

And don't get started on legacy English. ;-)

October 10, 2006 11:23 AM

 
Blogger www.totsyssoft.com said...

Maybe you can find a unix coder who has laboured on all the hard parts and now gives it away with make && configure. Then you'll have more time to preach.
Blah blah blah

October 11, 2006 3:41 AM

 
Blogger bowerbird said...

> After reading John Gruber's
> response over at Daring Fireball
> (and not figuring out
> how to post comments there)

let us know when you figure that out,
won't you please? thanks. :+)

-bowerbird

October 11, 2006 12:30 PM

 
Anonymous Anonymous said...

It would be hard to dispute the benefit of a consolidated implementation paradigm. And while the the performance canard inevitably pops-up, it's only a distraction. As far as resource issues are concerned, Apple has never been in a better financial position to enact its technical agenda, and in the end, the only one who can put the bell on the cat.

-Joel

October 11, 2006 1:44 PM

 
Anonymous Anonymous said...

Are you in sales and marketing by any chance? :-)

p.s. you forgot to drop the word "synergy" in there!

October 11, 2006 2:56 PM

 
Anonymous Anonymous said...

Rosyna,

You are mistaken on two points.  NSURL is indeed the preferred way to refer to a file in Cocoa these days, and has been for at least two years now.  You can expect future APIs to use NSURL * wherever they need to refer to an object in the filesystem (or on the network).

Also, how in the world did you get the idea that NSDocument was using an FSRef internally? Check NSDocument.h:  It has an NSURL ivar, _fileURL.  No FSRef.  

-jcr

October 11, 2006 11:13 PM

 
Anonymous Ben said...

[jcr]
... NSURL is indeed the preferred way to refer to a file in Cocoa these days, and has been for at least two years now. You can expect future APIs to use NSURL * wherever they need to refer to an object in the filesystem (or on the network).


The AppKit Release Notes mention NSURLs and NSDocument/NSDocumentController:
"... The new methods all consistently deal in NSURLs, replacing the melange of paths and URLs that previously existed."

However, the PDFDocument class (available in Mac OS X v10.4 and later) has writeToURL: and writeToFile: methods, which is superfluous.

It's not clear how NSDocument keeps track of the document's file, at least not when NSURLs are used.

setFileURL:
"... The default implementation of this method just records the URL so that the default implementation of fileURL can return it."

setFileName: (Deprecated)
"... The Finder also keeps track of open documents and their associated files. When a user moves or renames a file in the Finder that corresponds to an open document, the Finder calls setFileName: with the new filename."

October 12, 2006 5:15 AM

 
Anonymous Anonymous said...

About the inaccurate documentation mentioned in the original post: Please do report problems like that using the doc feedback form at the bottom of each doc page.

Those forms are processed by real humans, and errors like that are addressed pretty quickly these days, now that monthly updates are provided for Apple's reference doc.

October 12, 2006 4:39 PM

 
Blogger Wil Shipley said...

Those forms are processed by real humans, and errors like that are addressed pretty quickly these days, now that monthly updates are provided for Apple's reference doc.

I neglected to say, in my original post, that I have filed bug reports on all of these issues at bugreport.apple.com, my VERY BEST FRIEND in the whole world. I file about a bug a day now, I think. I didn't know about the documentation link but I did file a documentation bug in RADAR for the Keychain function I savage, #4727670, but it was helpfully marked as "duplicate" and "closed" so I have no idea if/when it'll be fixed. (My disfavorite feature of RADAR: the stupid "duplicate" status that cuts me off from ever being able to take my workarounds out of my code, and from interacting with engineers as they trace the issue down. If you're going to mark me as "duplicate", at least leave the bug "open" until you fix the problem, and then I'll have a clue!)

In general I don't like yelling about bugs; instead I always report them directly to Apple, because as I developer I know how much it annoys me when a customer gets on a forum and insults me instead of just telling me what's bugging her so I can fix it.

But sometimes I *do* need to raise issues in public. Some changes are bigger than a single bug report -- they need a groundswell of support from the developer community in order to get Apple behind them.

October 12, 2006 5:18 PM

 
Anonymous Rosyna said...

jcr, simple. NSDocument uses an NSFileSpecifier to track document files. NSFileSpecifiers are just classes with a single ivar, an FSRef.

Furthermore, you cannot say just because a class has a specific ivar that it is being used. But with specific regard to NSDocument, it has a private data member, which can be anything or a collection of anything. Most of the CoreImage classes also have this, a single private (opaque) ivar that points to a piece of data defined by the class that can change at any time without worrying about backwards or future compatibility.

October 13, 2006 1:17 AM

 
Anonymous Anonymous said...

@drew thaller

I love legacy english so np.

Unfortunately "Various and sundry" is one of the classic examples commonly used to demonstrate what is known in the grammarian's world as "Redundant Pairs".
Redundant Pairs, is one of the three types of redundancy; the other two are "Redundant Modifiers" and "Redundant Categories".

As far as the "shades of meaning" you ascribe personally to words, they really do not affect the definitions agreed upon by lexicographers:

Various = from the French varieux, from Latin varius "changing, different, diverse."
Sundry, however, usually refers to "an indeterminate number" but that is only as a noun. As an adjective it means... various.

So you should argue it in reverse:

"Various" emphasizes the diversity, while "sundry" emphasizes the multiplicity.

It is nevertheless an incorrect argument.

Redundancy means that you say the same thing again but in different words or different forms of the words.

You wrote:

"For various and sundry really good technical reasons, no. :-)"

If we re-wrote your sentence as:
-
For sundry really good technical reasons, no.

or

For various really good technical reasons, no.
-

It would lose not a shade of meaning.


Common usage has never made grammatically incorrect usage correct.


Common redundant pairs:

full and complete
true and accurate
hopes and desires
each and every
first and foremost
any and all
various and sundry
basic and fundamental
so on and so forth


Common non-redundant pairs:

questions and problems
tried and true
willing and able

Sorry Wil ;; on polluting your comment section with this stuff.

October 15, 2006 12:46 PM

 
Blogger Wil Shipley said...

Sorry Wil ;; on polluting your comment section with this stuff.

Actually this is probably the most interesting post here, including the ones I made. I'm fascinated by English as well.

However, I think your analysis of redundancy omits an important facet, which is _why_ people intentionally use redundancy: as a semi-poetic device to to underscore their point.

It's a device I use all the time, eg: "George Bush is vile. He is evil." (But, not, I guess, "live" or "veil".)

October 15, 2006 3:10 PM

 
Anonymous Anonymous said...

Umm... I'm not so sure that "any and all" is redundant - "any" is very diffrent from "all" --- "any of them could have done it" is not the same meaning as "all of them could have done it"; nor is it the name as "any and all of them could have done it".

Consequently, substituting [someCollection anyObject] for [someCollection allObjects] is (independant of other factors) an error... or something like that :-)

October 15, 2006 3:11 PM

 
Anonymous Anonymous said...

Bush is not evil - just sometimes misguided. Evil is intending to do harm, and enjoying it. I'm not sure I could call most of Al Quida evil either - sick, malignant, dangerous, and worthy of summary execution, but not evil (the difference being that they think that they are doing the right thing - even though they are, to the rest of us, obviously very wrong about that). War rarely involves true evil - each side tends to think that they are doing right, but one or both sides is - at least partially - in error.

P.s. - as a devout Darwinian, I believe that we must impose severe penalties on severe misbehavior in order to keep the system evolving in a nominally "positive" direction... so, yeah, torturing and killing terrorists (er - real ones, not punks who walk thru the airport security line with a smart-ass T-shirt on) is OK in my book.

October 15, 2006 3:31 PM

 
Blogger Wil Shipley said...

I define "evil" as "acting only in your self-interest, to the extent that others are gravely harmed."

Every personal, professional, and political decision Bush 2 has made has been for himself and his friends. The entire notion that American lives are inherently more valuable that Iraqi lives demonstrates this admirably, but there are countless examples. (Hey, let's put oil lobbyists in charge of the EPA! Let's weaken the definition of "grain-fed" and "organic" so my giant farmer lobbyist buddies can poison my people with their crap! Let's label a war hero as "unfit to command" when I was a draft-dodger who didn't even show up to my cushy mail-delivery job with the air national guard! Let's have a war on drugs when I was a junkie! Let's put the army in charge of rebuilding Iraq and ignore my own State Department's plan for a rational reconstruction. Let's...)

I don't inherently have trouble with punishing criminals. This isn't about that. It's about (a) us losing our humanity by practicing torture, and (b) us being unable to distinguish who IS and ISN'T a terrorist, and deciding to find out by just torturing everyone.

If there are really as many real terrorists as there are people in American gulags, then we need to revise our definition of terrorist and/or take a good hard look at our foreign policy.

October 15, 2006 4:40 PM

 
Anonymous Jack said...

What about us "crazy" folk who use a "non-c" language? Cocoa just does not work out at all. So, we can NOT benefit from any of the effort. Oh well!

October 15, 2006 5:09 PM

 
Blogger Wil Shipley said...

Jack: A language is a tool. A framework and the conventions surrounding it are more powerful tools.

Pick the language for the toolset you want to use. Picking a language in a vacuum is like saying, "I like in America but I chose to only speak Esperanto, so I'm S.O.L.!"

October 15, 2006 6:37 PM

 
Anonymous Jack said...

Cocoa requires objective-c. I did not pick the language, Apple did. I agree you pick the best tool for the job, but since there is really only one tool, that's not actually a choice. Since Apple has chosen to bless objective-c, that's it. Either other languages create a "bridge" for injecting objective-c into their environment or no access to Cocoa. Good luck selling that.

Most "other" languages have some mechanism for including (and usually building) standard c libraries. That's because c is a "known standard" that everyone has agreed on, more or less. Nobody I know of has any built-in mechanism for including objective-c, except for c based languages compiled with gcc. Consider that every OO implementation has it's own runtime, many their own compiler, and none of them are designed to work with others.

OK I admit that I am biased. I write the IDE for Marten, www.andescotia.com a visual, data-flow, programming language editor. But, I've written a Cocoa bridge, and I know what it takes, and I know where it breaks.

I have no problem with other languages, I really love them all. Every language on the Mac can produce a standard platform binary "Carbon" app but only obj-c makes a "Cocoa" app. And that's exactly the response I get, "I'm S.O.L.", except never quite so demurely. I think everybody loses if there is "only" obj-c.

We would love to offer integrated Cocoa access in our product, but don't hold your breath. The bridge works, but it's not obj-c or Prograph so it feels weird. The burden of writing "wrappers" falls to each developer, so they are all different. If Apple chooses to only use objective-c, and that eliminates us from serious consideration, what do you call that?

October 15, 2006 9:17 PM

 
Anonymous Graham Cox said...

Some interesting points from both sides of the "argument". Carbon isn't that bad in my view, but then I spent years programming in the Mac toolbox and Carbon, so I guess it's a case of familiarity not breeding contempt. Nowadays I LOVE Cocoa for its nearly-right architecture and high-level goodness. But I'm not afraid of Carbon for some tasks if that's how it has to be - in fact the way to handle this is to create a Cocoa object that wraps up useful Carbon functionality so you only have to do that dirty work once - from then on you have you useful high-level Cocoa object. Trading this code is a whole new community in itself.

I don't think there is much merit in the "argment" - it's horses for courses. Carbon is a foundation - if you want high level but can't bring yourself to learn Obj-C, build an OO framework on top of it (been done many times, including my own MacZoop, may it rest in peace). However, Obj-C only LOOKS tricky with all the brackets and stuff - in practice I found it was very quick and easy to assimilate. So despite defending Carbon for what it is, I would also say that developers who are resisting Cocoa should give it more of a chance - it's a lot simpler than it looks. Also, these days if you want your product to look modern you'll need to render using Quartz, and there's really no easier way to do that than from Cocoa.

"Hell, you should have seen how tough it was to get some people to even give up using Quickdraw, for god's sake."

I'm sure, it seems crazy... and yet - if only Cocoa could do what Regions can do. I mean trivial union, intersection, difference methods on closed paths. C'mon Apple, let's see this added to NSBezierPath. It would knock down the only remaining good reason not to use QD.

October 16, 2006 10:01 PM

 
Anonymous Anonymous said...

I disagree with your terminology... that's merely malignantly narcissistic. True evil is a whole nuther' level below that...

Punishing criminals is inherently dehumanizing... even if they aren't really criminals, and it's just supposed to be some silly psych experiment in the basement of Stanford U. ( http://www.prisonexp.org/ ). The harsh reality is that somebody has to take that ugly hit to the soul so the rest of us can lead lives unencumbered by that inherent dehumanization... (and yes - consummately professional police, correctional officers, and soldiers do take a hit, even if they strive not to let it effect their behavior on-the-job).

You are absolutely right, though, about telling the difference between who's a terrorist and who aint'. AFAIAC DHS has a LOT of explaining to do...

Most (but, not all) of the folks in the gulag aren't there because of foreign policy, they are there because of religious zelotry (principally fundamentalist wahabisim); even if we had never gone over there they would continue to attack us because a) we exist; and b) their religion allows for nothing but themselves to exist (everything else must be destroyed in jihad). And yes - there are a lot of them!


_________________________________________________

Therefore, for the good of your soul, Apple, stop punishing us with Carbon! (... like how I brought that back on topic? ;-) )

October 16, 2006 10:31 PM

 
Blogger Wil Shipley said...

I'm sure, it seems crazy... and yet - if only Cocoa could do what Regions can do. I mean trivial union, intersection, difference methods on closed paths. C'mon Apple, let's see this added to NSBezierPath. It would knock down the only remaining good reason not to use QD.

I don't claim experience with QD, so it may not do what you need, but I think PDF (and thus CoreGraphics) has region logic built-in.

In PDF you would define a composite bezier path (made of several closed subpaths), and in one fill mode, if you wound a subpath clockwise it would subtract from the composite region, and if you wound it counterclockwise it would add. (Or vice-versa, but you get the point.)

This was in PDF since the beginning, if I recall -- it's been a LONG time since I wrote the PDF parser.

October 17, 2006 1:28 AM

 
Anonymous Jack said...

Cocoa uses a dynamic, object-oriented language (Objective-C) by design, and was the inspiration for modern languages like Java.

OK cool. Let's just assume, in theory, that the whole OS is written in Cocoa and obj-c.
Are you all telling me you don't ever want to use a "modern" language? Are you sure it's a good idea that nobody can use a "modern" language? What advantage is that?

October 17, 2006 2:02 PM

 
Blogger Wil Shipley said...

Jack: There's no advantage to only working with one language, except it's a hell of a lot easier for Apple to support. That's not an advantage for us programmers per se, but it translates into them having time to implement actual cool features and fix bugs and make things faster.

We supported Java before, nobody liked it. The truth is, Java on the desktop is kind of a nonstarter, so far. This may change in the future, but regardless of my feelings towards Java (I hate it) it's hard to argue that it's taken over the world of desktop apps.

Note that the runtime in ObjC 2 (with garbage collection) is WAY more flexible than ObjC (classic) in terms of allowing multiple languages to co-exist under one roof, but I think the details are under NDA. So, we're both going to get our way, in the end.

October 17, 2006 2:39 PM

 
Blogger Kilroy Trout said...

Wil,

FSRefs, QuickTime gobblygook, why did you have to resurrect these nightmares in an old Mac software developer?

Now where is that bottle of Tequila...

October 27, 2006 10:56 PM

 
Blogger Artie MacStrawman said...

Regarding how to post comments at Daring Fireball: one doesn't.

December 23, 2006 11:54 PM

 

Post a Comment

<< Home