Category Archives: Software Development

Bias towards shipping

A few weeks ago a colleague said that I had one of the stronger biases towards shipping that he had seen. I am pretty sure he meant it as a compliment, and I took it that way anyway. In my work, I am highly influenced by the Steve Jobs quote, “Real artists ship”, and I often say that our work as product developers is to ship and get better at shipping.

That being said, it’s sometimes hard for me to remember that in my personal projects. Back in January, I started an iPhone app to help me stay motivated to stick to the Paleo dietI blogged about that back then, and said:

I’ve been working on a way to do this (mostly to scratch my own itch), and will have more to say on that soon.

My plan was to ship in April. But, a trip to New Zealand and my duties at work left me with less time to devote to it. The project languished from March to about a week or two ago when I got a message from Apple that my right to the name PaleoViz would be revoked if I didn’t submit a binary.

With that kick in the pants, I went into full ship mode. Removing features, fixing bugs, finding what I hope are elegant solutions to thorny user interaction problems. The biggest decision I made in the bias to ship was to abandon having my own online photo sharing and am just using Twitter for that (for now). PaleoViz was reduced to its essence — a photo food journal app for paleo dieters.

I expect it to be approved by the end of August, if you are a paleo blogger and want a review copy, let me know.

GPL for Portfolio Pieces

On my Atalasoft blog I wrote that I thought programming job applications should become more portfolio oriented, and

that means working code that I can run, and access to all of it. Probably the best thing is a running website and a github repository. If the software is hard to run, then a video walkthrough or screenshots are acceptable substitutes.

To learn more about what I want this to be like, I am starting to prepare a portfolio for myself.

One issue is that it’s important to decide what license to apply to it. If you don’t decide, then the default license is all rights are reserved, and the code isn’t very usable by anyone else. Since this code is meant just to show others what I’ve written, I guess it could be ok, but I actually don’t mind others using the code.

Another issue is that I intend to post code for projects that will be for sale (iPhone apps). If I put a permissive license, like BSD or MIT on it, then I run the risk of anyone being able to compete with me with my code and my permission.  But, not open sourcing real code would be against the whole point of making a portfolio. Fake code doesn’t represent programmers well.

I think Zed Shaw has the right idea about using GPL, not so much the why, but in what ways GPL helps authors.

Using GPL, I get these benefits

  1. Use of the code requires attribution
  2. You can’t put GPL code in the App Store
  3. I can still grant BSD licenses on a case by case basis, and perhaps only on portions. To do so requires the developer to get in touch with me
  4. Likewise, I can still choose to sell or give away commercial licenses, if I am contacted.

To me, github and the entire idea of a portfolio is to make code more social. GPL helps maximize that.

SOPA Solution: Block Congress

A few weeks ago, I tweeted:

 

Today, I was wondering how easy that would be, and spent about 10 seconds in google to find that Congressional staffer IPs have been outed by Wikipedia for editing their boss’s pages:

Wikinews contributors have discovered that members of the United States Congress or members of their staff have recently been making questionable edits to Wikipedia.

[…] In one instance, Wikinews found that someone with one of the IP addresses, 143.231.249.141, began to edit the Wikipedia article for Steve Austria, the Republican representative for Ohio’s 7th congressional district.

[…] Another individual, with the IP address 75.187.63.132, also removed the allegations of plagiarism from Austria’s article in February. The individual removed what they called “Politically Motivated BS” from the article of Deborah Pryce.

To confirm, I put the first IP in an IP Locator tool and got this:

From here you can figure out an IP range to target, and then it’s trivial to serve them different content (perhaps giving them a taste of what SOPA IP blocking would be like). I leave that as an exercise to the reader.

To be effective it needs to be done by sites that Congressional staffers actually depend on or on enough smaller sites to get media attention.

Sunday is Pushday

I believe that making a commitment public helps you stick to it. So, because I want to make more of my private code public on github, I am treating each Sunday as “Pushday”.

On every Sunday, I am going to push something, no matter how small, into github. If I have nothing useful, I will write something new. My only restriction, is that the push must be useful (a bug fix, new feature, a minimal project).

Today, I am taking one of the JavaScript files from my pet project ASCIIMatic, and making it available under BSD. ASCIIMatic takes simple descriptions of diagrams, translates them into dot, runs it through the dot interpreter to get a set of drawing instructions, and then draws the result in ASCII art. The vision is to make it easy to include UML diagrams in your source code comments, but it has a long way to go to do UML. Right now, it can do a some simple box and arrow diagramming.

Today, I pushed ascii-drawing.js, which implements a simple ASCII drawing surface with draw line, rect, and text commands.

It is available under the asciimatic-scripts project on github.

Objective-C Warning: ‘class’ may not respond to ‘-message:’

If you get a warning of this form:

warning: ‘Class’ may not respond to ‘-message:’
(Messages without a matching method signature will be assumed to return 'id' and accept '...' as arguments.)

There are a few things to check:

  1. Make sure that you spelled the name of the message correctly, and that you matched all of the types of all of the arguments
  2. If the message is called in the same class, then the message you call must be higher in the class or the message must be declared in the header
  3. If you are calling it from a different class, then the message must be declared in the header

Remember, the declaration of the message must match the actual message signature. For example, if you have a message like this:

-(void) message:(NSString *)value
{
// message implementation
}

Then, you need this in your header:

-(void) message:(NSString *)value; // don’t forget the semicolon

Get iPhone programming tips in your inbox with my Beginner iPhone Programming Tips newsletter.

Inception Movie Review from a Data Security Perspective

This is part of my ongoing series of extremely limited perspective movie reviews:

Previous Reviews:

Like the previous ones: SPOILER ALERT. Inception is the kind of movie where you might not want to know anything going in. So, please don’t read further if you don’t want any part of the plot spoiled.

In Inception there’s an elite team that can infiltrate your subconscious while you’re dreaming and extract secret information while your defenses are down. The plot of the movie is set up when someone asks them if they can do an “inception” where they plant an idea into a victim’s subconscious. Since the idea is detrimental to the victim, they must somehow convince him that it is his, and that it will be good for him to follow through with it.

Obviously, extraction directly relates to data security, and the attacks and defenses that are discussed in the movie have analogous ones in the security world. An obvious one is a honeypot and how it relates to the labyrinths that the dream architects construct.

[Honeypots] computers run special software, designed to appear to an intruder as being important and worth looking into. In reality, these programs are dummies, and their patterns are constructed specifically to foster interest in attackers.

Creating a honeypot is like “taking two minutes to create a maze that takes a minute to solve.”

Things like a “militarized subconscious”, “totems”, and “forgers” have counterparts in the network security world.

But, what about “inception”? After thinking about it for a few minutes, I remembered this story from WWII. In 1943, the Allies wanted to convince the Germans that they would attack through Sardinia, not Sicily, so:

The idea, very simply, was to get a dead body, equip the dead body with false papers, and then drop it somewhere the Germans would find it

[…] And it was an elaborate creation: the fictitious Major Martin was equipped with ticket stubs, keys, a religious medal, letters from an imaginary father and fiancee, and unpaid bills. Cholmondeley and Montagu thought that the more convincing his personal story was, the more likely the Germans would be to believe the ruse. And along with the personal items, he carried carefully faked letters hinting that the Allies were planning to invade Greece and Sardinia, not Sicily.

But, this is not a computer attack. While the idea of data tampering is nothing new, I haven’t heard of a tampering attack with the intent to mislead someone into making a bad decision. Data tampering is often used to gain access for another purpose or to cover up tracks (e.g. log tampering).

But, I suspect that this is a real threat as well. One so good, that it  often goes undetected.

Making Xcode’s Build and Analyze much better

A few weeks ago, I discovered that the Build and Analyze in Xcode didn’t find as many problems as scan-build did directly when you turn all options on.

On the Clang analyzer site, they have instructions for setting up the latest analyzer so that Xcode would use it. Unfortunately, the settings that find some very common problems are off by default. Here are instructions for getting the Clang analyzer to run with those options on.

To make sure you are doing it right, make your project completely succeed a Build and Analyze, then comment out a release that you are doing in any dealloc. Rerun Build and Analyze — it should not find this problem.

Then,

1. If you don’t have it, download and untar the latest checker
2. Make a script file called full-analyze-clang and put it in the checker’s bin directory, with this code

#!/bin/bash
CLANG="`dirname $0`/clang"
CLANG_CMD="\"$CLANG\" -Xanalyzer -analyzer-check-objc-missing-dealloc -Xanalyzer -analyzer-check-objc-missing-dealloc -Xanalyzer -analyzer-experimental-internal-checks $*"
eval $CLANG_CMD

3. From the Terminal, run this command to make the script executable

chmod +x full-analyze-clang

4. In the checker root directory, there is a script called set-xcode-analyzer. Run it like this:

sudo ./set-xcode-analyzer --use-checker-build=FULL_PATH_TO_CHECKER/bin/full-analyze-clang

replacing FULL_PATH_TO_CHECKER with the path to the checker directory where you untarred it in step 1.

5. To test, Run the Build and Analyze on the project you set up with the bad dealloc. If it finds 0 errors, this could be because something is wrong with the way the script is set up.

Important parts if this stops working with a future clang:
The set-xcode-analyzer expects a directory or a full-path to clang. It checks to see it has a full-path by looking for a path ending with “clang”, which works for us because our script’s name ends in “clang”. If they change this script in the future, this might not work

Build and Analyze just reports a successful build if our script reports errors — it’s just looking for the existence of result files. You might need to add logging lines to the script to figure out what’s going on. Log to a full-path because the current directory is inside Xcode’s path somewhere when the script is run.

Use something like this as your last line to debug:

eval $CLANG_CMD > `dirname $0`/cl.out 2> `dirname $0`/cl.err

Get iPhone programming tips in your inbox with my Beginner iPhone Programming Tips newsletter.

Xcode 4

There are two great things about Xcode 4 that mean I had to download and try it right away

  1. It runs side-by-side on the same machine as Xcode 3.2
  2. It uses the exact same project file format

This hasn’t been the case in the last few major upgrades from what I remember, and I was really glad to read that in the release notes.

My first impressions after an hour or so:

Putting the view in Assistant mode (Opt-Cmd-Enter) is really nice. Shows the .m when editing the .h and vice versa. When editing an .xib file, it shows the ViewController.

Keeping the Interface Builder dialogs organized as parts of a single window is also a big win. It’s confusing having them as free dialogs. Specifically, having the parts of the main dialog (File’s Owner, First Responder, View, etc) as just part of the chrome (and always visible) is nice.

I think I’ll really like the Log Navigator — especially for debugging. I am constantly clearing the console, but this saves each one in a separate view so I can keep them around.

The Project Navigator is so much cleaner (replaces the Groups and Files tree). It always felt like Xcode just stuffed everything there instead of organizing it elsewhere in the GUI. However, it took me a long time to figure out where Build Settings were (click the project name in the Project Navigator), but it seems obvious now.

Build and Analyze still doesn’t find dealloc omissions of retained property releases. I would like the default to be set to finding these.

Some keyboard shortcut changes. The GUI is so radically different, so this is excusable. For the most part, things are the same.

Get iPhone programming tips in your inbox with my Beginner iPhone Programming Tips newsletter.

LGPL and the iPhone

The entire point of the LGPL is to give closed source applications access to an open-source library in a way that maintains the freedom of the library. Specifically, the user of the application must be able to change the LGPL library and have the application use it.

To achieve this, the LGPL gives you a few options for compliance. I am not a lawyer, but I believe that all of them are incompatible with iPhone apps delivered legally through the AppStore. I will lay out my reasoning here:

Section 4d gives two options for compliance: Either provide the source or object files suitable for relinking (4d0) or use a shared library (4d1).

Shared libraries are out, because Apple does not permit their use in the AppStore.

Section 4d0 is a little more complicated. For an application targeted to a PC or Mac, it would be easy to comply by just packaging up your .o files, the LGPL library, a makefile, and documentation. However there are subclauses in section 4 that make me believe that there is no way to comply on the iPhone. Specifically:

Section 4d0 not only specifies that you must convey the minimal source and object files, but you must also provide terms. Together they must “permit the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work”

Section 4e requires that you provide instructions that allows someone to change the LGPL portions, create a new combined work and install and execute that combined work.

To do either of these things, the user would need to be enrolled in Apple’s developer program which costs $99/year. These terms are implicitly part of your terms, meaning that it’s basically equivalent to you charging $99/year for the user to make modifications.

The GPL allows you to charge what you want to convey the Combined Work to begin with, but section 10 of the GPL (which I think still applies to the LGPL) states:

You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License.

Now, obviously, being able to develop isn’t free, but those costs are not as directly related to exercising your rights as the $99/year is. Specifically, you cannot create an executable or put in on a device without the certificates you obtain from the program.

Here’s an alternative view from Huy Zing — I think the “Spirit of the LPGL” section in the blog is actually required by the text of the license.

Here’s a discussion on the cocos2d forum about switching away from LGPL. They tried adding to the license at first and then moved to an MIT license.

I haven’t been able to find any official ruling on gnu.org, so I will write them and see if they can add to their FAQ (as they did for Java when it was an issue).

Get iPhone programming tips in your inbox with my Beginner iPhone Programming Tips newsletter.

Understanding EXC_BAD_ACCESS

A couple of days ago, I wrote about how to debug a crash that reports EXC_BAD_ACCESS. One thing I didn’t cover is what EXC_BAD_ACCESS means, which I’ll try to do now, as it will clear up a lot of the questions I’m getting about the previous blog.

The description here is a high-level way of thinking about it. The details are quite a bit more complicated, so I’m simplifying it.

On the iPhone (and most modern OS’s), your application is given memory as you need it. The memory is given in chunks that are bigger than your request, and then the unused parts are parceled out over the next few requests.

When you deallocate an object, that chunk can’t be returned to the OS right away. It has to wait until all of the memory in the chunk is deallocated.

Inside all of this memory is a complex data structure that is maintained by the alloc and dealloc messages to organize how each part of the allocated memory is being used. The pointers you hold are just part of that datastructure (where the object is), there are other parts that are only used by the allocator.

What EXC_BAD_ACCESS is saying is that you did something that caused a pointer (yours, one internal to the iPhone, or one that the allocator is using) to be dereferenced and that memory location isn’t inside one of the chunks assigned to your program.

This could be because

  1. The pointer used to point to memory that was ok, but its chunk was deallocated.
  2. The pointer is corrupt.

The line of code that your app crashes on is not the root cause of the problem. The problem in #1 is whatever line of code caused the premature deallocation, and the problem in #2 is whatever line of code corrupted the pointer.

Your goal in debugging this is to make the problem line of code be flagged by either the compiler or debugger.

If you do that, then fixing it becomes a lot easier.

Of the two possible problems, #1 is far easier to find. It’s almost definitely because you didn’t use retain/release correctly and there where either too many releases or too few retains.

Do this:

  1. Run Build and Analyze. Make sure you fix or understand every single error it flags. I personally have 0 Build and Analyze errors in every project I have and I go out of my way to keep it that way. If I ever get a false positive, I figure out how to make Build and Analyze understand what is going on, so that it doesn’t flag it.
  2. Run scan-build with all checks on. This isn’t built in, so if you’re in a hurry, skip this for now. scan-build is the project that Build and Analyze is based on. It can be run with much more thorough settings.
  3. Set up Xcode so that it never deallocates. Instead, it turns objects into Zombies that complain if they are used. See Tip #1 on this post for instructions.

If you have a clean Build and Analyze and no Zombies complain of being accessed, and you still get EXC_BAD_ACCESS, then it’s a good bet that you are not accessing deallocated memory. It’s not a sure bet, because the iPhone SDK gives you access to the C library which uses a different kind of allocation, which you could be using wrong.

For #2, your task is harder. If a pointer is corrupt, there are lots of possible reasons

  1. The pointer could have never been initialized.
  2. The pointer could have been accidentally written over because you overstepped the bounds of an array
  3. The pointer could be part of an object that was casted incorrectly, and then written to
  4. Any of the above could have corrupted a different pointer that now points at or near this pointer, and using that one corrupts this one (and so on)

If this is the situation you are in, then these are the things that will help

  1. Enable Guard Malloc (Tip #2) – this makes the datastructure that represents the allocations much more sensitive to corruption. You need to use the enhanced features of the debugger to get anything out of it (explained in the tip).
  2. The line of code that triggers the crash is a clue to what pointer is corrupt. If you move it around it might be able to help you narrow down the point of corruption.
  3. If all else fails and you are desperate, try Valgrind.

Once one of these methods gives you a different problem (either a warning or another EXC_BAD_ACCESS), don’t worry that it seems completely unrelated to your original problem — that’s a common problem with corruption.

Also, random changes to your program may make this problem “go away” — it’s not really fixed, though. Your corruption or early deallocation is still there, but it’s not triggering an EXC_BAD_ACCESS. Its effects could be far worse, however, so it’s a good idea to try to keep reproducing it until you are sure you addressed the problem.

Things to remember

  1. Corrupting a pointer doesn’t immediately trigger EXC_BAD_ACCESS. Neither does using a corrupted pointer unless it’s specifically now pointing to memory that isn’t mapped to your application.
  2. Deallocating an object doesn’t immediately release the memory to the operating system — only when the chunk is unused, can it be returned.
  3. The line of code that is triggering the EXC_BAD_ACCESS might not be the problem. It can be a good clue, but don’t assume the problem is this code.