themorgue.org

I’ve been using SourceForge to host some of my open source projects for a while. One thing that has been annoying to me is typing in passwords whenever I do any CVS commands. It hasn’t annoyed me enough to automate the process though. However, I recently started working with a new editor called Editra, and I’m creating a project management plugin that handles source control which needs automatic logins.

I figured that there must be an automatic login procedure for repositories in SourceForge because there had to be someone that was annoyed enough with password prompts to implement it. I searched the documentation on the site, but that was fruitless. While there is a lot of documentation on SourceForge’s web site, it was just too much to sort through to find the one little piece that I needed. So I started searching the web for a simple solution and came across this page.

The solution is rather easy, especially for someone who already has SSH keys. Simply log into your SourceForge account, go to http://sourceforge.net/account/editsshkeys.php, and paste in your ~/.ssh/id_dsa.pub key.

I’ve decided to team up with a friend of mine to create a new application that will be implemented on the web. This is the first commercial venture that I’ve ever taken up personally, but it sounded interesting enough to take a crack at it. My friend actually brought up the idea. He has the domain knowledge and industry connections, but not enough programming experience to pull it off on his own. I have a vague idea of the domain and quite a bit of programming under my belt, so it seemed like a good pairing.

In making the decision on how to implement this software, it was decided to do it in a web application as a service rather than as shrink-wrapped software. This actually has benefits for our company as well as the customer. The customers don’t ever have to upgrade, worry about backing up data, and the interface will be platform independent. Our company benefits by having a constant stream of income from the subscriptions.

Since all of my prior programming work has been in existing company environments, or open source projects hosted on SourceForge, I never had to set up a complete development environment of my own. Since this is my company and I get to pick whatever I want, I thought I’d share what I think is the ultimate web application development environment.

Source Control

I’ve been using CVS both at work and in my open source projects for years. However, Subversion is getting very popular nowadays. After reading the documentation, it seems to solve a lot issues with CVS and has a nice programmable interface.

Bug Tracking

While I’m doing the programming and know that there won’t be many bugs to track, I figure I might as well have some bug tracking software around just for kicks. ;) I didn’t really find any free solutions that got my attention, so I decided to look at some commercial products. I finally decided on FogBugz. I’ve been reading Joel’s website for quite a while, and have also read his books. I figure I’d see if he’s really all he’s cracked up to be and try out some of his software. Plus, it’s not really that expensive.

Backups

If you’ve read my previous blog entries, you’d know that I am a big fan of backups. I’ll still be using SuperDuper! for my mirrored backups, but I’ll definitely want to have remote backups as well. For remote backups, I’ll probably use a combination of Mozy and rsync.

Web Application Software

If you do any work in web applications today, Ruby on Rails is probably on the short list of tools to use. Well, I’ve been using Python as my main language (for personal projects) for quite a few years now and haven’t found any of the benefits of Ruby to be significant enough over Python to take the time to learn it; so I think I’ll stick with Python. In the Python world there are three major players in the web application framework ring: Zope, Django, and Turbogears.

I was a big fan of Zope many, many years ago. In fact, I started learning Python just for the purpose of using Zope. I even wrote several websites using Zope for work and for personal use. It offers a huge number of features and management capabilities, but it is a big pill to swallow. When you get into the world of Zope, you pretty much have to take-it-all, or leave it. While I do still like Zope, I think I’m going to pass for this project.

Django was the next candidate. I don’t really know that much about Django, but I just didn’t feel right when I read the documentation on the website. I also don’t like the fact that they use their own template language (aren’t there enough already). So on to Turbogears…

Turbogears has a philosophy that I really relate to: take a bunch of really good, open source tools and combine them into something cool. The documentation on the website was nice, and there is a book as well. It also supports various types of page templates (much like my LaTeX processing framework, plasTeX)! Overall, Turbogears looks like the framework for me.

Database

I’ve used PostgreSQL for various little projects throughout the years mainly because I liked having transactions, sub-selects, and free software. Up until recently, this was the only free database that had all three of those. However, even though PostgreSQL seems to be the more feature-filled database, MySQL is clearly more popular, and better supported on most web hosting sites. MySQL does support transactions and sub-selects now, and even if it didn’t there isn’t much direct manipulation of SQL in Turbogears, so it wouldn’t make much difference. In addition, FogBugz requires MySQL, so I was going to have to have a MySQL server up anyway. Due to all of these reasons, I’m going to pick MySQL for my new project.

Conclusion

There is still more software that I need to evaluate the business end of things like accounting, customer databases, etc., but the development environment appears to be set at this time. While many of these tools are new to me, and it will take some time to get up to speed, I’m looking forward to trying out a combination of tools that I got to hand pick for my own business.

I’ve been working on improving my workflow in programming lately. This is mainly due to the fact that I now work from home. Working from home poses some new problems since I don’t have a 1Gb connection to my network resources anymore. In the process of rethinking my processes, I decided to play around with FileMerge, one of Apple’s developer tools. FileMerge enables you to visually see the difference between two files and interactively merge them into one final form. This is a very common operation when using CVS and you have a conflict with a push from another developer, or you just want to see the differences between two file revisions. Because I use it in conjunction with CVS so much, I thought I’d add some CVS features to FileMerge.

The result of this work is cvs-opendiff. It’s called cvs-opendiff because Apple supplies a utility called opendiff which invokes FileMerge from the command-line, and cvs-opendiff invokes this command along with various CVS commands to do its work. The feature list for cvs-opendiff is shown below.

• Compares two files like diff
• Compares the current working file with the current revision in CVS
• Compares two file revisions or dates
• Continues running until FileMerge is exited, or the window containing the file comparison is closed

This last feature is the key feature since the opendiff command exits as soon as it finished invoking FileMerge. This behavior is often undesirable when merging files especially when the merge is invoked by a third tool that expects this behavior.

I’d love to hear your comments and suggestions for cvs-opendiff. I’m planning on starting to work with subversion soon, so I imagine I’ll be updating it to work with that eventually.

Guido van Rossum posted an update on Python 3000, the next big Python release that is due out in alpha form by the end of the year. What’s interesting about this release is that it is throwing backwards compatibility to the wind. It will definitely be fascinating to see how that turns out.

Overall, I’m pretty excited about this new release. The Python crew is trying to do away with a lot of cruft that has been building up over the years. One of the biggest changes is in printing and formatting. There will no longer be a print statement. Printing message will now be done with print() function. The % string formatting character will also be removed. The most interesting part though is the new formatting strings. They are no longer in the form of “%s” or “%(name)s”. In fact, they look more like a page templating language now with ways to reference attributes, mappings, and sequences.

One thing I’m looking forward to is unicode everywhere. Working on plasTeX has shown me just how vigilant you have to be in programming to keep all of your strings in unicode form. This should make things easier.

I’ve been working through Dave Thomas’ CodeKata for the past few weeks. I get to about one or two a week. I just finished number six which has been the most fun so far. Dave’s told about his solution in Ruby that was 25 lines and took 1.5s on his 1GHz machine. Since I’m more of a Python fan than a Ruby fan, I thought I’d take a shot at his numbers. Here is my 7 line solution.

words = {}
for word in open('wordlist.txt'):
    key = ''.join(sorted(list(word.strip().lower())))
    words.setdefault(key, []).append(word.strip())
for v in sorted(words.values()):
    if len(v) > 1:
        print ' '.join(v)

This could be shortened to 5 lines, but isn’t quite as readable.

words = {}
for word in open('wordlist.txt'):
    words.setdefault(''.join(sorted(list(word.strip().lower()))), []).append(word.strip())
for v in [x for x in sorted(words.values()) if len(x) > 1]:
    print ' '.join(v)

On my 1.83GHz MacBook, this took 0.56s.

While I didn’t get to go to PyCon this year, I read Guido van Rossum’s impressions. It’s quite apparent that the use of Python in One Laptop Per Child project is going to be driving the development of Python in the following months. One paragraph on Guido’s blog that really excites me is shown below.

The software is far from finished. An early version of the GUI and window manager are available, and a few small demo applications: chat, video, two games, and a web browser, and that’s about it! The plan is to write all applications in Python (except for the web browser), and a “view source” button should show the Python source for the currently running application. In the tradition of Smalltalk (Alan Kay is on the OLPC board, and has endorsed the project’s use of Python) the user should be able to edit any part of a “live” aplication and see the effects of the change immediately in the application’s behavior. (A versioned document store will make it possible to roll back disastrous changes.) This is where Krstic wants my help: he hopes I can work magic and implement this feature for Python. I got started right away during the conference, with a reimplementation of python’s reload() function that can patch classes and functions in place. Even this small component still has a long way to go; a checkpoint of the work in progress is checked into subversion as part of the Py3k standard library. That’s not where the rest of my OLPC work will show up; they use GIT for source control, so I will get to learn that.

While I think the changes to Python in the past couple of years have been a bit esoteric (and a bit disheartening), having the ability to update live code would make me as happy as a peach orchard hog! It might even be the tipping point to get Blaine, my friendly neighborhood Smalltalk apologist, to start using Python.

After working on a few glitches for a plasTeX user this week, plasTeX is now being used for the documentation of the itools package for Python. Hopefully, this exposure will help out with plasTeX’s adoption for other packages and eventually the documentation at python.org itself.

I submitted a paper to PyCon 2007 this year about my open source framework for processing LaTeX documents, plasTeX. Unfortunately, the reviewers thought that it was too specialized and esoteric. While it is fairly specialized, I don’t see it as being esoteric. Especially since the official way to write Python documentation is in LaTeX. I was hoping to increase interest in plasTeX so that it might some day replace latex2html (a Perl program) as the markup generator for Python documentation. This would eliminate installing Perl as a pre-requisite to writing documentation for Python.

I guess I’m not too worried about it at the moment. plasTeX is still very young and needs testing before it can be used in production environments. By PyCon 2008, it should be ready for action!

While it’s always possible to script an Applescript Studio Application using GUI scripting, it’s pretty clunky to constantly be referring to windows and buttons. The good news is that it’s not too difficult to add custom commands to your application. However, you will need to know some Objective-C to do this.

(more…)

I was looking for a way to start a DVD (or DVD folder) in Apple’s DVD Player using Applescript such that it skips the menu, trailers, etc. It turns out that this is fairly simple for most disks because most DVDs start the main feature at chapter 1 of title 1. Here is the code.

tell application "DVD Player"
    play dvd
    set title to 1
    set chapter to 1
end tell

For discs that don’t fit this model, you can store away the proper title and chapter numbers and look them up on a per DVD basis.

Update: I’ve found out that this does not work with all discs. However, I was using this trick with DVDs ripped by MacTheRipper with the DVD navigation limitations removed.