The little picture in the location bar is a favicon (Favourites Icon). When you visit a site and bookmark a page IE 5.5 (or higher) requests a file called favicon.ico which is a 16x16 image that you see.
MSIE looks automatically for "favicon.ico" in the root directory of your site, this can be irritating as sites without them tend to get a lot of 404 errors generated from it. You can specify a different location using the <link> element, which (as usual) goes in the <head> section of the document:
<LINK rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
Any URL can be used here; it doesn't have to be named "favicon.ico" or be in your site's root.
Other browsers do not look for an icon by default, but the above <link> will tell them (if they support it) to fetch one.
In fact, Mozilla (at least - perhaps others too?) even allows the icon to be in other graphic file formats, not just Windows icon format (make sure you update the value of the type attribute if you don't use a windows icon file). Internet Explorer only supports windows icon files though.
Also see http://www.pcmag.com/article/0,2997,s%3D1478%26a%3D4753,00.asp for FavOrg, a utility that automatically refreshes favicons in the favorites menu for Internet Explorer for Windows.