More FTP stuff

Mon, Mar 3, 2003

Bitch, bitch, bitch. Okay, Chris, what do you want in the FTP upstreamer? I don't think that Radio's upstreamer deletes either. Here is what I can do easily:

  • CD to a directory. Throw if the dir isn't there.
  • [done] Ensure a directory. Try my best to make sure that a directory (and the path up to it) exists.
  • Delete a directory. The server will probably return an error if there are files in it. I will throw.
  • Get the current directory.
  • Move up a direcotry.
  • [done] Upload a file into the current directory.
  • Download a file in the current directory and return a stream. It would be easiest to buffer the entire file in memory.
  • Delete a file in the current directory.
  • [done] Return a text string with the current directory listing.
  • Call SYST and return whatever the system gives us. This might help detect the type of system on the other end.

The only thing that isn't easy and that will probably take quite a while to get right is parsing the output of the directory listing. I don't have a bunch of servers to test this against to make sure it will work.

As for the upstreamer itself, I'd rather have it not try and sync the stuff on the server (and hence delete stuff) because I'd be afraid of loosing data. For instance, I upstream into the root of my site and I have all sorts of subdirs off of that. I'm setting up a weblog for Rachel on eightypercent.net and her root will be a subdir. I don't want to clobber her stuff whenever I upload my site.

Jackpot, NV

Sun, Mar 2, 2003

Rachel and I took a quick trip to Jackpot, NV this weekend. Watch this space -- stories to come. It was Craptacular!

Non passive FTP support

Sun, Mar 2, 2003

I coded up non passive FTP support for my FTP code. It only took me ~30 mins. I've handed that off to Chris so perhaps it will show up in BlogX at some point. If not, I may write a standalone utility and release that.

It was pretty easy to send the PORT command and start accepting connections. I had to mess around a little bit to get it right so I wish that the documentation around using sockets was better. For instance, there is no short sample that shows creating a socket, binding it to an address and then accepting connections.

BTW, my router (Linksys wireless/4 port) works with both passive and regular FTP connections. It must be monitoring the control connection and (a) rewriting it with the public FTP address and (b) opening the port through to my machine. These little boxes are just amazing.

Rachel to blog?

Sat, Mar 1, 2003

I'm trying to convince my wife that she needs to start a web page to tell funny medical stories. Stay tuned!

I-831

Fri, Feb 28, 2003
NOW, THEREFORE, BE IT RESOLVED, That the citizens of the State of Washington do hereby proclaim that Tim Eyman is a Horse's Ass.

Democracy in action. I saw this a while ago but just ran across it again from Jon Stahl reached via GeoURL. (I really thought about changing my GeoURL pointer to Boise, but ... Boise is boring!)

PASV ftp

Thu, Feb 27, 2003

My FTP code only supports PASV ftp right now. The reasoning is that I'm behind a NAT firewall and my server isn't. PASV works much better in that configuration. Chris is having problems getting my code to work and he is blaming this.

FTP generally has two TCP connections: a control connection and a data connection. The control connection is pretty straigtforward. You connect to port 21 and start sending messages back and forth. Some of these commands ask to open a data connection -- to upload a file, download a file or get a directory listing.

There are two ways to establish the data connection:

  1. Normal FTP: you can have the server establish the connection to your machine. You tell the server that you are listening on a certain port and that he should connect to you there.
  2. Passive FTP: you establish the connection to the server. You ask the server for another address/port and you initiate the connection.

In a world with network address translation (NAT) it is a lot easier to connect from your protected machine out to the network than it is to have another machine connect (on a new port) to you. This is why I did passive FTP. Doing normal FTP shouldn't be that hard if this is indeed the problem that Chris is having.

Oh yeah, and if both client and server are behind NAT, you are screwed. That is unless one of the NATs is smart enough to understand FTP and translate stuff along the way. HTTP can work better here since everyone understands it and it all happens on one socket. However, HTTP servers aren't set up for this type of thing without wacky extensions like WebDAV or the frontpage extensions.

D60 obsolete

Thu, Feb 27, 2003

Well, I knew it was coming, but my *ahem* brand new D60 is now superceded by the new Canon 10D. It has the one thing that I really really want -- a better AF system. At least I know that mere mortals won't get their hands on them for many months yet.

New tripod

Mon, Feb 24, 2003

Ordered one of these bad boys today. I can't wait. I'm also going to order one of these pretty soon too.

My old tripod had a bent leg and was super heavy. With this combo I'll be set to carry the tripod on longer hikes. Things should start warming up soon and I can start exploring the environs around Boise.

WWCDD?

Mon, Feb 24, 2003

Michael Winser suggested that I should think about taking the same approach that citydesk uses. They do two things differently than Radio:

  • They use a CRC instead of a byte for byte comparison. Apparently they use this instead of filetimes altogether. I could compute the CRC (or do the comparison with the file on disk too, I guess) by wrapping the stream that the transform is writing to and doing the CRC/comparison on the fly
  • They also post a list of files and CRCs as an xml file on the server. This makes it easier to synchronize files across machines

I definitely want to enable the cross machine case, and file times do seem a little fragile to me. This is something I'll have to think about a little more.

Two way synchronization is really hard though? What happens if the same file changes on both the client and the server? How do you handle deleting a file? (Do you care about deleting a file?)

Another problem comes up when you have multi-party authoring and syncing and that is with respect to item numbers. Right now I have an item number count that I store in a config file and update every time I create a new entry (it is currently at 49). ChrisAn solves this problem with a good old fashion GUID (I have a bunch cheap if you need them ;) I might be able to switch over without killing my RSS feed and perma links if I just treat the item ID as a string.