Poisoned NULL byte (and friends)

[cgay]

This conversation was maybe 14 years ago (there's no year in the timestamps)... maybe it's time to do something about it in v2.0 of the http libraries.

May 17 14:45:27 <andreas>	Carl: I think you need a generic mechanism to make sure an attacker cannot trick his way through protection mechanisms by using different names for the same file.
May 17 14:45:38 <andreas>	Are you aware of the poisoned NULL byte?
May 17 14:45:53 <cgay>	no, but it sounds bad :)
May 17 14:47:29 <andreas>	Imagine a web server written in a language that keeps explicit track of string lengths. Like Dylan. Running on top of a OS with a system interface inspired by C, such as UNIX oder Windows, where strings are NULL-terminated.
May 17 14:47:49 <andreas>	Now imagine what happens when you handle a request like foo.exe%00.txt
May 17 14:49:05 <cgay>	you pass that string to the OS open() and all hell breaks loose?
May 17 14:49:52 <andreas>	What usually happens is that you map from filename extensions to MIME types, and from there to request handlers.
May 17 14:50:14 <andreas>	Say that .exe maps to application/cgi-bin, and .txt maps to text/plain.
May 17 14:50:34 <andreas>	YOu have that CGI there, which of course contains your database passwords.
May 17 14:51:28 <andreas>	The %00 gets converted into a NULL byte, which is a legal part of the string. SO your code thinks the user requests a txt file, and calls the appropriate handler.
May 17 14:51:46 <andreas>	Which calls open() on the string it gets, and delivers the contents to the user.
May 17 14:52:04 <andreas>	Of course, at this moment, the .txt is lost, and your server happily delivers the application binary.
May 17 14:52:04 <^self>	i see.
May 17 14:52:18 <cgay>	ah
May 17 14:53:17 <andreas>	WIndows has a couple more gotchas like that. There once was the problem that you could append $00 to a filename to do some magic, and that $00 was stripped by the OS before opening the file.
May 17 14:53:42 <andreas>	So you need a generic pathname canonicalization mechanism to protect against this problem in a systematic manner.
May 17 14:54:06 <andreas>	This needs to be carefully worked out for every platform, including all of these gotchas.
May 17 14:54:49 <cgay>	i c.  do you have a good source for this info?  otherwise i'll just search for 'poisoned null byte'
May 17 14:55:33 <cgay>	i wonder if CL-HTTP has already done something like this...
May 17 14:55:59 <andreas>	I have explained all there is to know about that. The hard part is coming up with a design that allows for central maintenance of platform rules, so the user code doesn't have to worry anymore.
May 17 14:56:15 <andreas>	You need to fight the whole class of bugs, not just this single instance.
May 17 14:58:22 <cgay>	yup
May 17 14:59:49 <andreas>	I've seen that you did well on preventing directory traversal, btw. :)
May 17 15:00:46 <andreas>	There are web servers you can fool with %2e%2f%2f
May 17 15:00:58 <andreas>	Err... %2e%2e%2f that is.
May 17 15:01:20 <cgay>	which must be ../
May 17 15:02:01 <--	hannes has quit ("-> cccb")
May 17 15:02:10 <cgay>	I'll make a note of your comments and try to get to that soon