The often deranged postings of yet another hacker, pretending to be an Astronomer, pretending to be a hacker who has written a book or two for O'Reilly Media.
The idea of ADS to KML came up over morning coffee on the last day of the .astronomy meeting, and by the close of the conference I had most of it hacked together...
What am I talking about? A lot of papers on ADS now have links to the SIMBAD database for further information on the objects they discuss. For instance I was recently a co-author on an exo-planet paper which links to the relevant objects in SIMBAD...
The mashup at that point was obvious. Do an ADS query and look for all the papers with links into SIMBAD, then do a series of follow-up queries on SIMBAD and grab all of the objects mentioned in the papers. Then generate a KML file of your publication history, which you can either display directly in Google Sky, or embed into a Google Maps for Sky as I've done above.
Of course not all papers reference objects, and not all papers with objects have SIMBAD links, especially older papers. None the less, having run my script to generate a KML file for several colleagues now it actually gives a fairly good representation of their research interests.
You could imagine several ways to extend my quick hack. If you had a large enough group of astronomers, and therefore a large enough number of papers, you could produce heat maps of the sky instead of using simple push pins. You could cross-correlate your own publications with that of a group or institute where you're thinking of applying for a job, or the publication output of a survey team with the footprint of their survey...
Comments welcome, but yes, I already know it's an interesting but essentially pointless hack. I mean other comments...
So we've been playing around with Google Maps for Sky, as opposed to Google Earth for Sky which has been around a few months longer, since December last year. But yesterday Google got round to releasing the 'official' Sky Maps site. It's pretty cool, and I'm glad I never got round to fiddling with opacity sliders for multiple wavelengths, because they've done it better than I would have done.
After my previous efforts with Google Maps for Sky and the native Maps.app on the iPod touch, Ryan Scranton at Google pointed me in the direction of a Google Code project that was attempting to provide a more friendly user interface to the online Javascript version of Google Maps on the iPhone and the iPod touch.
It turned out to be a fairly trivial project to modify the code to use the new Sky maps, so on yet another wet Sunday afternoon, I did...
The results aren't quite as slick as the native version, but it was a lot less effort. You can drag the map around the screen using two fingers, although unfortunately you can't use the pinch action to zoom in and out. Crucially since it's just the normal maps with extra Javascript wrappers you can make use of the Maps API to add placemarks and overlays and other interesting things...
However getting used to using two fingers to drag the map around instead of one is surprisingly hard, it's very anti-intuitive for some reason. I'm also not entirely convinced it runs fast enough, at least for my purposes, it seems much less responsive and much more jerky during scrolling than the native Maps.app application. It's a great hack, but not the solution to my problem. But don't take my word for it, if you've got an iPhone or an iPod touch you can try it for yourself.
The release of the new Google Maps API for Sky got me thinking. Google Maps, and Maps for Sky, doesn't really work very well inside Safari.app on the iPhone or the iPod touch. The maps in the browser aren't set up to trigger on the correct Javascript events, and as a result scrolling and zooming just doesn't work correctly. However there is a mobile Maps.app for the iPhone, which not only scrolls, pans, and zooms properly, but is also a whole lot faster than the browser based version.
So I didn't really want to play with the browser based version of Google Maps on my iPod, I wanted to play with the mobile version. So the obvious thing to do would be to hack the mobile version of Maps.app to support the Sky. It's just a different tile set, shouldn't be too hard?
There is nothing obvious in the plist files associated with the Maps application or the GMM.framework itself, but this is only really to be expected. So I pulled out my hex editor of choice, I recommend Hex Fiend by the way, to see what I could be done. Initially, and unfortunately somewhat optimistically, I thought that it should just be a case of tracking down the strings containing the URLs where Maps pulls tiles from the remote server, and changing these to point to the Sky tile server instead. But after poking around for a while inside the GMM.framework this didn't look like it was going to be too easy, at least without decompiling the code. However I did discover references to a SQL database which the application was using as a tile cache for downloaded tiles. Pulling this database off the phone an into SQLLite on my desktop let me examine the schema, which has three separate tables,
images(zoom int, x int, y int, flags int, length int, data blob); version(version int); index1 on images (zoom,x,y,flags);
The x, y and z were self explanatory. That's just the x and y position of the tile at zoom level z. The flags looks to be whether the tile is a map (flag is 2) or satellite (flag is 3) tile, the blob was presumably the tile itself and length was obviously the length of the binary blob in bytes. Probably...
The obvious strategy at this point was to download all the tiles, and yes, I mean all the tiles, from the Google Sky tile server, and poke them into the tile cache database in the appropriate places to fake out the Maps.app application into thinking it didn't have to go to the web to download those tiles after all.
Oddly enough the maps tiles served by Google Maps for Sky are JPG files, and the maps tiles in the cache database are PNG files. The JPG's also seem to be 256×256 pixels in size, while the tiles in the cache file are 128×128 pixels in size. So we had a bit of an issue here. I could either take the thousands of downloaded tiles, convert them to PNG, and split them up into smaller tiles, or I could just try throwing them into the cache and see what happened. Want to take a guess at which one I chose to do first?
Pulling the eatblob.c out of the SQLite tutorial, I hacked together a quick script to push all 5,461 tiles (zoom level 0 to zoom level 6) into the cache database. Copying the new enlarged database back onto my iPod, I restarted the iPod and fired up Maps.app but things didn't exactly go as planned. Predictably I didn't get the mapping between the x and y's of the Sky tiles and the Map tiles quite right. In fact I got my x and y's round the wrong way in one of my scripts. Second time was the charm though...
Basically at this point I've got Google Maps for Sky working on my iPod touch under Firmware 1.1.2. This hack will presumably also work on the iPhone since my Maps.app was taken off a Jailbroken iPhone in the first place, after all my iPod touch didn't ship with the Maps application in the first place...
Despite all this there are still predictably some problems with the current implementation. First and foremost, the Sky map only occupies the upper left quarter of Earth map when zoomed out all the way out. The best explanation I can come up with for this is simply the fact I haven't yet cut my "big" tiles into quarters. If generate 4×(128×128) tiles out of each of my current 256×256 tiles at each of the current zoom levels then I reckon I'll fill the map.
Of course even with this solved there are two fundamental problems which means that this project is almost entirely an intellectual exercise. The current lack of zoom levels in the Sky data means than when you zoom in far enough you eventually drop back down onto the Earth. For instance the SMC is located somewhere over Marrakech, and if you zoom in far enough sub-Saharan Africa starts to poke through the star field. This could probably be lived with except for the fact that the only way to find the SMC in the first place is do a search for Marrakech, after all the search utility still thinks it's looking at the Earth's surface.
The other major problem is that the Maps.app application doesn't actually have the Maps API, the very thing I need so that I can manipulate the map at the very basic level of even adding a placemark. There are presumably hidden hooks into the application, and maybe those will appear in the Apple SDK due in February . But there still isn't any word on the licensing provisions for the SDK. Who knows if I can get access to it, especially if there is a issue surrounding the digital signatures rumoured to be needed for an "official" application. Finding the money needed, possibly a lot of money, for the SDK could be an interesting proposition. That's even if those hooks exist in the first place.
Anyway, the easiest thing for you to do if you want to replicate my hack is to download the hacked database file (29MB) I generated and copy it into /var/root/Library/Caches/MapTiles/. Make sure you make a copy of your existing MapTiles.sqlitedb database before dropping the new one ontop of it, you'll probably want to back out of the hack at some point. After copying the database file into the right place, simply reboot your iPhone or iPod touch. At that point it should all just work...
Source Code
If you're crazy enough to want to waste your time playing around with this I've attached the source code that will let you rebuild the MapTiles.sqlitedb database directly from the Sky tile servers. Hopefully the guys at Google won't be too cross, as I doubt the handful of people crazy enough to go off and duplicate this stunt will add any significant load to the tile servers...
We've know this was coming for a while, but not when. News of Google Maps for Sky leaked out last week on the Google Maps API forum, quickly followed by the official Google announcement on Friday.
You can create a Sky map in the same way you did your plain old regular map, all you have to do is pass it the correct mapType. So something like this will get you a handle to a Sky Map,
var map = new GMap2(document.getElementById("map"), { mapTypes : G_SKY_MAP_TYPES });
However if you want to do anything to the map, the coordinate mapping isn't pretty. If you've got your RA and Dec in decimal degrees, you'll need to fudge them like this to push a marker pin into the sky,
var point = new GLatLng( dec, -ra + 180 ); var marker = new GMarker(point); map.addOverlay(marker);
This initially had me very confused, not only is it back-to-front, it's not what you have to do to your RA and Dec to plot it in Google Earth. There you have to subtract 180 from your RA value before throwing it into your KML file. Which of course brings me to KML support, there doesn't seem to be any right now...
The lack of KML support is a bit disappointing, it'd be nice to be able to take the KML network link that has the live feed of event messages flowing across the eSTAR network, and throw it directly into Maps. But right now that isn't possible. No worries though, this is an early release in every sense of the word. It's coming, I'm sure I can count on the Googlers.
With that in mind, it'd be daft for me to rewrite the code to throw the live events into Google Maps, because if KML support is coming, and surely it is, I'd just be wasting my effort. So I decided to sit down and PLASTIC-enable Maps instead. A few months ago I did the same for Google Earth, although obviously I had to take a slightly different tack this time around while turning Google Maps for Sky into so called "tupperware".
As before I've written a small PLASTIC application, a facade, which registers with the PLASTIC Hub as normal and listens for ivo://votech.org/sky/pointAtCoords messages. These are PLASTIC messages telling interested applications to "point at" an RA & Dec. When such a message is passed through the Hub, say by CDS Aladin, it is forwarded to the facade application, which then injects this into a boiler plate HTML file. The facade application then exposes this HTML file via an embedded web server. If you point a web browser at this end point you get a map with the marker plotted at the relevant RA & Dec.
Additionally when you send a PLASTIC message into facade application the place mark it generates has a link which, when clicked, will call back to the facade application's embedded webserver and allow you to send a PLASTIC message back to the Hub. This means there is bi-directional control, both in and out, of Google Maps with PLASTIC.
I've uploaded the source code again, but be warned, this is proof-of-concept stuff and when you're dealing with Perl that can get pretty torturous. Especially since I based it on the previous proof-of-concept code I wrote for Google Earth and never got round to fixing up.
There are obvious problems with my implementation, after all it was just a wet Sunday afternoon's work, but most of these are fixable with a little refactoring. However it does work, and uses a lot of the same concepts as my previous effort. This hints strongly that it really is time I got round to writing that general use PLASTIC module for Perl that I've been talking about for a year or more...
Unlike their image labeler application, this appeal is founded quite firmly in self interest. It makes sense, if you want people to be able to find your house or office, and you can actually see people doing it, and in the process radically improving the Google's geo-referencing and search results...
The new Google street side view layer in Google Maps
Of course this isn't new, after all Amazon A9 did this back in 2005, but neither was online mapping applications when Google Maps launched. The question isn't whether they're were first, it's whether their interface is so much better than what's gone before (yet again) that they corner the market. That, and from the sounds of things, whether this gets integrated into Google Earth.
However while the guys over at O'Reilly seem to be able to see this on the Google Maps pages right now, it looks like the rest of us will have to wait for the official announcement during Where 2.0 later today. I certainly can't see the new features and from the comments on their post it doesn't look like I'm alone on that one.
Update: I can confirm that adding &gl=us (via Lifehacker) to the URL does indeed allow you to use the new features, at least from the UK, and from what I remember it blows the interface for the long defunct A9 service out of the water. The way you can click and drag around your location is actually pretty slick.
Publications for Allan, A. as KML
