DOS/Windows command line escaping

Sun, Dec 5, 2004

Here is a Raymond-esque tidbit that I wanted to get out there.  I couldn't find anything on Google and had to figure this stuff out via trail and error.

Okay -- you think you know how to escape stuff on the DOS/Windows command line?  Probably not.  Let's start with a simple python script to show us what we've got:

C:\test>type dumpargs.py
# a simple program to dump the incoming arguments
import sys
for arg in sys.argv[1:]:
    print repr(arg)

Now lets run some tests and see what it spits out:

C:\test>dumpargs.py "hi there"
'hi there'

C:\test>dumpargs.py "hi \" there"
'hi " there'

So far so good. It looks like we know how to escape a double quote sign ("). But what about the pesky percent sign? DOS/Windows/Command/CMD uses this for variable substitution. Let's try a bunch of stuff and see what we get:

C:\test>set foo=bar

C:\test>dumpargs.py "hi there %foo%"
'hi there bar'

C:\test>dumpargs.py "hi there \%foo\%"
'hi there \\%foo\\%'

Hmmm, it looks like the backslash doesn't work. Looking around, it looks like the caret symbol (^) is the way to go. Lets try that out.

C:\test>dumpargs.py ^%foo^%
'%foo%'

C:\test>dumpargs.py "hi there ^%foo^%"
'hi there ^%foo^%'

Well, it looks like that works as long as it isn't in quotes. I also found another wacky case:

C:\test>dumpargs.py "hi there \" ^%foo^%"
'hi there " %foo%'

C:\test>dumpargs.py "hi there \" \" ^%foo^%"
'hi there " " ^%foo^%'

Very odd -- it looks like the caret does work as an escape character as long as there is an odd number of escaped double quote characters before it in the string. The parser here is obviously not a simple as one might first think. Lets try ending the string before we quote the percent sign:

C:\test>dumpargs.py "hi there "^%"foo"^%" bar"
'hi there %foo% bar'

C:\test>dumpargs.py "hi \" there "^%"foo"^%" bar"
'hi " there ^%foo^% bar'

We are getting closer, but that escaped quote character puts the parser into a wacky state. I finally did find a good solution:

C:\test>dumpargs.py "hi "^"" there "^%"foo"^%" bar"
'hi " there %foo% bar'

It turns out that you must replace the double quote (") with the following sequence: "^"". You must also quote the percent character (%) with this sequence: "^%". I think that everything else is okay inside of the double quotes. Very strange but true. I guess this is what 20 years of back compat get you.