tag:blogger.com,1999:blog-77349002024-03-19T09:44:49.642+00:00The Daily ACKThe often deranged postings of yet another hacker, pretending to be an Astronomer, pretending to be a hacker who has written a book or two for O'Reilly Media.Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.comBlogger1196125tag:blogger.com,1999:blog-7734900.post-2648217443334743872014-04-24T18:37:00.001+01:002014-04-24T18:37:30.773+01:00Crowdfunding the recovery of a lost spacecraft<div class="separator" style="clear: both; text-align: center;">
</div>
<i><div style="text-align: center;">
<i>This post was original published <a href="http://makezine.com/2014/04/24/crowdfunding-the-recovery-of-a-lost-spacecraft/">on the MAKE Blog</a></i></div>
</i><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-IElDWh1VEyk/U1jh96EgS_I/AAAAAAABkV8/_fbjbJQjj4I/s1600/Bl7yC7WIUAEz9_e.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://4.bp.blogspot.com/-IElDWh1VEyk/U1jh96EgS_I/AAAAAAABkV8/_fbjbJQjj4I/s1600/Bl7yC7WIUAEz9_e.jpg" height="280" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The <a href="http://en.wikipedia.org/wiki/International_Cometary_Explorer">ISEE-3</a> spacecraft</i></td></tr>
</tbody></table>
<div style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">The hackers behind the <a href="http://www.moonviews.com/" style="color: #005d8f; text-decoration: none;">Lunar Orbiter Image Recovery Project</a> have moved on to a different challenge. Not content with images, this time they want to <a href="http://www.rockethub.com/42228" style="color: #005d8f; text-decoration: none;">recover a whole spacecraft</a>.</span></div>
<div style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">The <a href="http://en.wikipedia.org/wiki/International_Cometary_Explorer" style="color: #005d8f; text-decoration: none;">ISEE-3</a> probe was launched in 1978. After completing it’s original mission—it was the first spacecraft ever to enter a <a href="http://en.wikipedia.org/wiki/Halo_orbit" style="color: #005d8f; text-decoration: none;">halo orbit</a> at one of the Earth-Sun <a href="http://en.wikipedia.org/wiki/Lagrangian_point" style="color: #005d8f; text-decoration: none;">Lagrangian points</a>—studying the interaction between the Earth’s magnetic field and the solar wind, it was repurposed—leaving its halo orbit—it was sent on its way to intercept Comet <a href="http://en.wikipedia.org/wiki/21P/Giacobini-Zinner" style="color: #005d8f; text-decoration: none;">Giacobini-Zinner</a> in 1985, and then Comet <a href="http://en.wikipedia.org/wiki/Comet_Halley" style="color: #005d8f; text-decoration: none;">Halley</a> in 1986 as part of the <a href="http://en.wikipedia.org/wiki/Halley_Armada" style="color: #005d8f; text-decoration: none;">Halley Armada</a>. Afterwards, left in a <a href="http://en.wikipedia.org/wiki/Heliocentric_orbit" style="color: #005d8f; text-decoration: none;">heliocentric orbit</a>, it was then used for investigations of coronal mass ejections until 1997 when it was decommissioned <a href="http://nssdc.gsfc.nasa.gov/nmc/spacecraftDisplay.do?id=1978-079A" style="color: #005d8f; text-decoration: none;">by <span class="s2">NASA</span></a>.</span></div>
<div class="p1" style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">However after the Comet Halley encounter in the 80′s the ISEE-3 was intentionally left in an orbit that would—eventually—bring the 35 year old spacecraft home, and if <a href="https://twitter.com/wingod" style="color: #005d8f; text-decoration: none;">Dennis Wingo</a> and <a href="https://twitter.com/KeithCowing" style="color: #005d8f; text-decoration: none;">Keith Cowing</a> have their way, it’ll return to a warm welcome from its creators.</span></div>
<div class="p1" style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">They’ve set up a <a href="http://www.rockethub.com/42228" style="color: #005d8f; text-decoration: none;">crowdfunding effort</a> to cover the costs of getting back in contact with the spacecraft, and ordering it to fire its thrusters one last time to put it into Earth orbit. The intricate trajectory necessary to make that happen—including a flyby of the Moon at an altitude of less than 50 km—has already been calculated by Robert Farquhar, the original mission design specialist from ISEE-3′s Halley encounter.</span></div>
<blockquote class="tr_bq">
<em>Our plan is simple: we intend to contact the ISEE-3 (International Sun-Earth Explorer) spacecraft, command it to fire its engine and enter an orbit near Earth, and then resume its original mission – a mission it began in 1978.</em></blockquote>
<div class="p1" style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">If successful ISEE-3 will spend its retirement as a platform for <span class="s2">citizen science</span>, with smartphone apps—and a <a href="https://twitter.com/ISEE3Reboot" style="color: #005d8f; text-decoration: none;">twitter feed</a>—giving students direct access to the instruments onboard the ageing spacecraft.</span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-S51-WFCtQUg/U1jh9xj-POI/AAAAAAABkWA/dXzbvdEYvjo/s1600/Bl3997_CMAABdWM.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://1.bp.blogspot.com/-S51-WFCtQUg/U1jh9xj-POI/AAAAAAABkWA/dXzbvdEYvjo/s1600/Bl3997_CMAABdWM.jpg" height="400" width="393" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The instrumentation carried by the ISEE-3 spacecraft.</i></td></tr>
</tbody></table>
<div class="p1" style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">While the spacecraft carries no imaging cameras, 12 of the probes 13 onboard instruments were still working <a href="http://en.wikipedia.org/wiki/International_Cometary_Explorer#Further_contact" style="color: #005d8f; text-decoration: none;">back in 1999</a>—the last time <a href="http://nssdc.gsfc.nasa.gov/nmc/spacecraftDisplay.do?id=1978-079A" style="color: #005d8f; text-decoration: none;">NASA</a> contacted the spacecraft—and it’d be a powerful tool in the hands of educators allowing amateurs and students access to instrumentation to measure plasma, high-energy particles and the magnetic fields in Earth orbit.</span></div>
<div class="p1" style="background-color: white; color: #333333; line-height: 25.200000762939453px; margin-bottom: 20px;">
<span style="font-family: inherit;">This is a great opportunity to put what is still world class instrumentation into the hands of the community. But orbital dynamics means that there’s only one chance to do so, and contact <a href="http://goo.gl/NB26i0" style="color: #005d8f; text-decoration: none;">must be reestablished with the probe</a> in late May or early June to ensure that the burn into Earth orbit happens during the correct window—and there are just 24 days left to <a href="http://www.rockethub.com/42228" style="color: #005d8f; text-decoration: none;">find the money to do it</a>.</span></div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com111tag:blogger.com,1999:blog-7734900.post-83831005349356901732014-01-05T19:22:00.000+00:002014-01-05T19:22:19.133+00:00The Board Room Hour<div style="text-align: justify;">
<span style="background-color: white;"><span style="color: #5e5e5e; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif;"><span style="font-size: 15px; line-height: 20px;">Back in October last year I was out in New York for the <a href="http://makezine.com/hardware-innovation-workshop/">Hardware Innovation Workshop</a> and <a href="http://makerfaire.com/">Maker Faire New York</a> where I took part in a panel discussion along with <a href="https://twitter.com/mbanzi">Massimo Banzi</a> and <a href="https://twitter.com/Jadon">Jason Kridner</a>—and chaired by <a href="http://makezine.com/">MAKE</a>'s <a href="https://twitter.com/dalepd">Dale Dougherty</a>—on what's in store for micro-controllers, and what the next generation of board could bring.</span></span></span></div>
<br />
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="253" scrolling="no" src="http://fora.tv/embed?id=19526&type=c" webkitallowfullscreen="" width="450"></iframe></div>
<div style="text-align: center;">
<i><a href="http://fora.tv/v/c19526">The Board Room Hour</a> from <a href="http://makerfaire.com/">Maker Faire</a></i></div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com63tag:blogger.com,1999:blog-7734900.post-41224059609582142292014-01-05T19:11:00.000+00:002014-01-05T19:11:30.563+00:00Hacking the CES Scavenger Hunt<center>
<em>This post was originally publish <a href="http://makezine.com/2014/01/03/hacking-the-ces-scavenger-hunt/">on the MAKE Blog</a> </em></center>
<center>
<em>and co-authored with <a href="https://twitter.com/sandeepmistry">Sandeep Mistry</a>.</em></center>
<center>
<br /></center>
<div style="text-align: justify;">
It has just been <a href="http://www.businesswire.com/news/home/20140102005966/en/CEA-Announces-iBeacon-Scavenger-Hunt-2014-International">announced</a> that at this year's <a href="http://www.cesweb.org/">Consumer Electronics Show</a> (CES) will feature a promotional scavenger hunt based around Apple's iBeacon technology. What if you could win the hunt, without ever having to go to CES? </div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-L-eFIEmfL1M/UsmtruUOnrI/AAAAAAAAglI/K8lM7d3_fBE/s1600/image.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://3.bp.blogspot.com/-L-eFIEmfL1M/UsmtruUOnrI/AAAAAAAAglI/K8lM7d3_fBE/s1600/image.png" height="400" width="322" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: small; text-align: start;"><i>What if you could win the hunt, without ever having to go to CES?</i></span></td></tr>
</tbody></table>
<div style="text-align: justify;">
Quietly introduced by Apple at <a href="http://www.apple.com/apple-events/june-2013/">WWDC</a> last year, iBeacon is a technology that allows you to add real world context to smart phone applications. Based around Bluetooth LE—part of the new Bluetooth 4.0 standard—it’s a way to provide basic indoor navigation and proximity detection.
As we talked about when we <a href="http://makezine.com/2014/01/03/reverse-engineering-the-estimote">reverse engineered the Estimote beacons</a>, there are three properties of an iBeacon that work together to create the beacon’s identity. These are:
</div>
<ul>
<li><b>UUID</b> — This is a property which is unique to each company, in most use cases the same UUID would be given to all beacons deployed by a company (or group).</li>
<li><b>Major</b> — The property that you use to specify a related set of beacons, e.g. in a retail setting all the beacons in one store would share the same Major value.</li>
<li><b>Minor</b> — The property that you use to specify a particular beacon in a location.</li>
</ul>
<div style="text-align: justify;">
The scavenger hunt is therefore a hunt for a number of beacons that will probably all share the same UUID and Major numbers, but will have different Minor numbers. Effectively, we're looking for a set of beacons.
However <a href="http://www.cesweb.org/Show-Floor/Maps">wandering the hallways</a> at CES hoping to get into the—approximate 100 foot range—of all of the iBeacons they've scattered across the show floor sounds like a lot of work.
CES has teamed up with <a href="http://www.radiusnetworks.com/">Radius Networks</a> who are providing the iBeacon hardware, and Marc Wallace—CEO and cofounder of Radius Networks—has <a href="http://www.businesswire.com/news/home/20140102005966/en/CEA-Announces-iBeacon-Scavenger-Hunt-2014-International">this to say</a> about the hunt,
</div>
<blockquote>
This is one of the coolest proximity-aware apps we have worked on. This is also one of the first, tangible applications that leverages iBeacon technology. And it is a great example of how iBeacon technology is not just about advertising as it is about bringing new and innovative solutions to the marketplace. We are very excited to be a part of it.</blockquote>
<div style="text-align: justify;">
Since they're using hardware from Radius Networks we can't just assume—as we could <a href="http://makezine.com/2014/01/03/reverse-engineering-the-estimote">with the Estimote hardware</a>—that we know the UUID of the beacons. However the identities of the beacons—all of the beacons—are somewhere where we can easily get our hands on them, the <a href="http://www.cesweb.org/Show-Floor/CES-Mobile-App">CES mobile app</a>.
Sure enough looking at the CES Android application—it's fairly easy just to <a href="http://apps.evozi.com/apk-downloader/">download the APK</a> without having to install—there are some hints there for us and <a href="http://stackoverflow.com/questions/12732882/reverse-engineering-from-an-apk-file-to-a-project">using a decompiler</a> it was fairly easy to find the details of the target beacons. </div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://makezineblog.files.wordpress.com/2014/01/screenshot-2014-01-03-13-50-54.png" style="margin-left: auto; margin-right: auto;"><img alt="The Minor numbers of the nine target beacons in the code of the CES mobile application." class="size-medium wp-image-364424" src="http://makezineblog.files.wordpress.com/2014/01/screenshot-2014-01-03-13-50-54.png?w=620" height="274" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: small; text-align: start;"><i>The Minor numbers of the nine target beacons in the code of the CES mobile application.</i></span></td></tr>
</tbody></table>
<div style="text-align: justify;">
The iBeacon UUID we're looking for is 842AF9C4-08F51-1E39-282F-23C91AEC05E, while the Major number—interestingly not actually needed and just ignored by the Android application—is 65000, while the nine beacons scattered throughout the <a href="http://www.cesweb.org/Show-Floor/Maps">CES venue</a> have Minor numbers from 65001 to 65009.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://makezineblog.files.wordpress.com/2014/01/ces2.png" style="margin-left: auto; margin-right: auto;"><img alt="The completed scavenger hunt—all nine beacons." class="size-full wp-image-364412" src="http://makezineblog.files.wordpress.com/2014/01/ces2.png" height="400" width="330" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: small;"><i>An almost completed scavenger hunt—with eight of the nine beacons already "found."</i></span></td></tr>
</tbody></table>
<div style="text-align: justify;">
Since we now know the identities of the beacons, it's trivial to finish the scavenger hunt without ever going to CES as it's actually fairly simple to build your own iBeacon hardware and <em>"fake"</em> the app into thinking you've found the beacons. To do that you can either <a href="http://learn.adafruit.com/pibeacon-ibeacon-with-a-raspberry-pi/overview">use a Raspberry Pi</a>, or <a href="http://redbearlab.com/ibeacon/">a Bluetooth LE board</a> like the Red Bear Labs <a href="http://www.makershed.com/ProductDetails.asp?ProductCode=MKRBL2&Click=156543">BLE Mini</a> board—<a href="http://developer.radiusnetworks.com/">Radius Networks</a>, the people supplying the hardware to CES, is even <a href="http://developer.radiusnetworks.com/ibeacon/ibeacon-development-kit.html">selling</a> a <i>"iBeacon Development Kit"</i> which would work just fine for our purposes. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
At which point—now you have your own iBeacon hardware—you can just go ahead and set the UUID, Major and Minor numbers of your beacon to each of the CES scavenger hunt beacon identities in turn, and then bring your beacon into range of your cell phone running which should be running the CES mobile app. Once you've shown the app all of the beacons, you'll have "finished" the scavenger hunt and can claim your prize. Of course doing that isn't legal. It's called fraud and will probably land you in serious trouble. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Of course it could be worse. If they are using Estimote hardware it'd be easy for someone to make the hunt impossible to complete. Because <a href="http://makezine.com/2014/01/03/reverse-engineering-the-estimote">as we've shown</a>, anyone with the Estimote SDK can modify the UUID, Major and Minor number of the Estimote beacons in the field. Which would have meant that the beacons deployed across the CES floor didn't work for the scavenger hunt anymore. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We talked about both of the ability to configure <em>"fake"</em> beacons, and the ability to disable beacon in the field—in our discussion of our <a href="http://makezine.com/2014/01/03/reverse-engineering-the-estimote">reverse engineering of the Estimote iBeacon hardware</a>. However, we didn't think we'd see something like this quite as soon.</div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com98tag:blogger.com,1999:blog-7734900.post-63055670087272559652014-01-05T18:55:00.000+00:002014-01-05T18:55:26.521+00:00The Snapchat Leak<center>
<em>This was first published <a href="http://radar.oreilly.com/2014/01/the-snapchat-leak.html">on the O'Reilly Radar</a></em></center>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://radar.oreilly.com/wp-files/2/2014/01/Screenshot-2014-01-01-22.42.12.png" style="margin-left: auto; margin-right: auto;"><img alt="The number of Snapchat users by area code." class=" wp-image-58816 " src="http://radar.oreilly.com/wp-files/2/2014/01/Screenshot-2014-01-01-22.42.12-620x453.png" height="292" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i><span style="font-size: small; text-align: start;">The number of Snapchat users by geographic location. Users are predominately located in New York, San Francisco and the surrounding greater New York and Bay Areas.</span> </i></td></tr>
</tbody></table>
While <a href="http://snapchatdb.info/">the site</a> crumbled quickly under the weight of so many people trying to get to the leaked data—and has now been suspended—there isn't really such a thing as putting the genie back in the bottle on the Internet. Just before Christmas the Australian based <a href="http://gibsonsec.org/">Gibson Security</a> published a report <a href="http://gibsonsec.org/snapchat/fulldisclosure/#obligatory-exploit-pocs">highlighting two exploits</a> in the <a href="http://www.snapchat.com/">Snapchat</a> API claiming that hackers could easily gain access to users’ personal data. Snapchat dismissed the report, <a href="http://blog.snapchat.com/post/71353347590/finding-friends-with-phone-numbers">responding</a> that,<br />
<br />
<blockquote class="tr_bq">
Theoretically, if someone were able to upload a huge set of phone numbers, like every number in an area code, or every possible number in the U.S., they could create a database of the results and match usernames to phone numbers that way.</blockquote>
<br />
Adding that they had various <em>"safeguards"</em> in place to make it difficult to do that. However it seems likely that—despite being explicitly mentioned in the <a href="http://gibsonsec.org/snapchat/index.html">initial report</a> four months previously—none of these safeguards included rate limiting requests to their server, because someone seems to have taken them up on their offer.<br />
<br />
<h2>
Data Release</h2>
Earlier today the creators of the now defunct <a href="http://snapchatdb.info/">SnapchatDB site</a> released 4.6 million records—both as an SQL dump and as a CSV file. With an <a href="http://news.cnet.com/8301-1023_3-57590968-93/snapchat-snapshot-app-counts-8m-adult-users-in-u.s/">estimated 8 million users</a> (May, 2013) of the app this represents around half the Snapchat user base. Each record consists of a Snapchat user name, a geographical location for the user, and partially anonymised phone number—the last two digits of the phone number having been obscured. While Gibson Security's <a href="http://gibsonsec.org/snapchat/fulldisclosure/#the-find_friends-exploit">find_friends exploit</a> has been patched by Snapchat, minor variations on the exploit are reported to still function, and if this data did come from the exploit—or a minor variation on it—uncovered by Gibson, then the dataset published by SnapchatDB is only part of the data the hackers now hold. In addition to the data already released they would have the full phone number of each user, and as well as the user name they should also have the—perhaps more revealing—screen name.
<br />
<br />
<h2>
Data Analysis</h2>
Taking an <a href="https://gist.github.com/aallan/8211440">initial look</a> at the data, there are no international numbers in the leaked database. All entries are US numbers, with the bulk of the users from—as you might expect—the greater New York, San Francisco and Bay areas. However I'd assume that the absence of international numbers is an indication of laziness rather than due to any technical limitation. For US based hackers it would be easy to iterate rapidly through the fairly predictable US number space, while <em>"</em><i>foreign"</i> numbers formats might present more of a challenge when writing a script to exploit the hole in Snapchat's security. Only 76 of the 322 area codes in the United States appear in the leaked database, alongside another two Canadian area codes, mapping to 67 discrete geographic locations—although not all the area codes and locations match suggesting that perhaps the locations aren't derived directly from the area code data. Despite some initial scepticism about the provenance of the data I've confirmed that this is a real data set. A quick trawl through the data has got multiple hits amongst my own friend group, including some I didn't know were on Snapchat—sorry guys. Since the last two digits were obscured in the leaked dataset the partial phone number string might—and frequently does—generate multiple matches amongst the 4.6 million records against a comparison number. I compared the several hundred US phone numbers amongst my own contacts against the database—you might <a href="https://gist.github.com/aallan/8211358">want to do that yourself</a>—and generated several spurious hits where the returned user names didn't really seem to map in any way to my contact. That said, as I already mentioned, I found several of my own friends amongst the leaked records, although I only knew it was them for sure because I knew both their phone number and typical choices of user names.
<br />
<br />
<h2>
Conclusions</h2>
As it stands therefore this data release is not—yet—critical, although it is certainly concerning, and for some individuals it might well be unfortunate. However if the SnapchatDB creators choose to release their full dataset things might well get a lot more interesting. If the full data set was released to the public, or obtained by a malicious third party, then the username, geographic location, phone number, and screen name—which might, for a lot of people, be their actual full name—would be available. This eventuality would be bad enough. However taking this data and cross-correlating it with another large corpus of data, say <a href="https://dev.twitter.com/docs/api/1/get/users/search">from Twitter</a> or <a href="http://en.gravatar.com/site/implement/profiles/">Gravatar</a>, by trying to find matching user or real names on those services—people tend to reuse usernames on multiple services after all—you might end up with a much larger aggregated data set including email addresses, photographs, and personal information. While there would be enough false positives—if matching solely against user names—that you'd have a interesting data cleaning task afterwards, it wouldn't be impossible. Possibly not even that difficult. I'm not interested in doing that correlation myself. But others will.Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com140tag:blogger.com,1999:blog-7734900.post-84140096099520539362013-08-15T00:00:00.000+01:002013-08-15T00:00:29.906+01:00The Trouble with ThingsAt the tail end of last month I was out in Portland at <a href="http://oscon.com/">OSCON</a> where I <a href="http://www.oscon.com/oscon2013/public/schedule/detail/29590">gave an Ignite talk</a> on the trouble with things...<br />
<br />
<center>
<object width="450" height="253"><param name="movie" value="//www.youtube.com/v/b-D2Gh2y0vM?version=3&hl=en_GB"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="//www.youtube.com/v/b-D2Gh2y0vM?version=3&hl=en_GB" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object></center>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com31Portland, OR, USA45.5234515 -122.676207145.1675755 -123.3216541 45.8793275 -122.0307601tag:blogger.com,1999:blog-7734900.post-31856766302389686212013-06-28T13:48:00.000+01:002013-08-05T01:09:45.323+01:00The Makers of Things<div style="text-align: center;">
<i>This post was first published <a href="http://makezine.com/2013/06/28/the-makers-of-things-2/">in Make magazine</a></i></div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/63923435" webkitallowfullscreen="" width="500"></iframe></div>
<div style="text-align: center;">
<i>The Society isn’t like any other engineering society. It’s a community united by a passion for making things and testing ideas. Their unparalleled devotion to their craft is evidence of a universal truth that’s relevant to us all; we learn only by doing.</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There’s something about Britain, it’s the land of God’s own hobbyists. But not only that, if there are more than two people in the country that are interested in a topic, they will feel almost compelled to for a club or society, and since this is Britain the club might well have been around since the early 1800′s or earlier. The bylaws will be arcane, and the committee structure labyrinth. It will inevitably serve tea and biscuits during meetings, rarely coffee. Because that’s how we do things over here.<br />
<div style="text-align: center;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/63927167" webkitallowfullscreen="" width="500"></iframe></div>
<div style="text-align: center;">
<i>Mike Kapp lives in a peaceful cul-desac, quietly shaping his house into a system of his invention. Cat-flaps, clocks and ghostly machines are the fruits of a lifetime of problem solving and inquiry. His experience holds a lesson for us all: curiosity is the key to craftsmanship.</i></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: justify;">
<div style="text-align: center;">
<div style="text-align: justify;">
<a href="http://www.themakersofthings.co.uk/">The Makers of Things</a> is a short film collection by <a href="https://twitter.com/anneholiday">Anne Hollowday</a>. It documents the work and workshops of the <a href="http://www.sm-ee.co.uk/">Society for Model and Experimental Engineers</a>, a sprawling organisation with members scattered all over the world. While today we <a href="http://makezine.com/2013/05/22/the-difference-between-hackerspaces-makerspaces-techshops-and-fablabs/">hacker spaces, maker spaces and fab labs</a>, back in the 1800′s they had the <a href="http://en.wikipedia.org/wiki/Learned_society">learned society</a>.</div>
</div>
<div style="text-align: center;">
<br /></div>
</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/62024423" webkitallowfullscreen="" width="500"></iframe></div>
<div style="text-align: center;">
<i>In the flickering light of a cinema Mike Chrisp watched a gentle Ealing comedy that would change his life. Now the linchpin of a world-renowned group of hobbyists and tinkerers, he builds models of the machines that inspired him all those years ago.</i></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: justify;">
Founded in 1898 by Percival Marshall, the Society seems to be almost a template, a model if you will, of one of these British institutions. Many were founded with royal patronage, and many still exist today. The Society has survived two world wars as well as the introduction of technologies that were not even be dreamed about when it was formed.</div>
<div style="text-align: justify;">
<blockquote>
<i>The world is full of of people that are making stuff in ways that were impossible a few years ago. There’s always going to be that mixture of people that think if you don’t make your own castings and machine them with files, cold chisels and hammers—you’re not doing it properly, and the other half that think the right way to do it is to use a laser—and increasingly things like three-dimensional printing, are coming along, and changing the way everybody works.</i></blockquote>
</div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/63923520" webkitallowfullscreen="" width="500"></iframe>
</div>
<div style="text-align: center;">
<i>Norman Billingham’s workshop has been pulled up around him over a lifetime. Chippings, clippings and filings fill a garage used to transform lumps of wood into beautiful pens and functional furniture. Although a scientist by training, Norman is the first to admit he’s always been a maker of things.</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Filmed almost entirely in the garage workshops of the members of the Society, the series of films evokes an atmosphere that’s possibly uniquely British, and it transports me back to my childhood. I can almost smell the wood shavings in the last of the four films.</div>
<div style="text-align: justify;">
<blockquote>
<i>If I had to live without it I could, but it’s something that’s been a part of me for a very long time. I’ve always been someone who makes things. That’s what I do, it’s always been a hobby. I’m a scientist by training—professionally—life’s work. But I’ve always been a maker of things.</i></blockquote>
</div>
As well as the film series, Anne also <a href="http://blog.newspaperclub.com/2013/06/26/the-makers-of-things/">created a newspaper</a>. It houses the extra stories and excerpts from the interviews that didn’t make it into the films, not because they weren’t fascinating, but because they needed a different medium to express them; as an accompaniment to the film series it acts almost like a projectionist’s commentary, or a more traditional programme you’d get at a theatre.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://blog.newspaperclub.com/2013/06/26/the-makers-of-things/" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="265" src="http://1.bp.blogspot.com/-zDOeE366jig/Uc2Cm80VgzI/AAAAAAAAY9g/gvp3VWWxhXs/s400/interviews-with-2.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Interviews with Model and Experimental Engineers</i></td></tr>
</tbody></table>
<blockquote>
<i>…some of the longer stories that my characters shared with me about specific points of their lives, or particular machines and methods didn’t make it into the films which I was pretty sad about. These were stories about machines invented under a veil of secrecy in the Soviet Union that someone had managed to build a version of, insight into a childhood with homemade toys and the reason why someone had spent their life building a particular type of locomotive. Tiny fragments of these tales made their way into the films but not in a way that did their stories justice.</i></blockquote>
I recently talked to Anne about her <a href="http://themakersofthings.co.uk/">series of films</a>, the newspaper, the story she was trying to tell, and the reaction to the piece both by the Society and others.<br />
<h4>
</h4>
<h4>
How did you get involved with the members of the Society?</h4>
<blockquote>
<i>I was on a bus going past Alexandra Palace in January 2012 and saw crowds of people streaming up the hill. It was more people than I’d ever seen going to Ally Pally before and a really diverse bunch of people – old, young, families etc, so I knew I had to find out what was going on. I looked it up online that night and turned out it was the London Model Engineering Exhibition. So I knew I had to go and see what was attracting such a huge bunch of people.</i><br />
<i><br /></i>
<i>The next day was the last of the three-day event and it was super busy again. We wandered round people selling tools, some trade stalls selling spare parts and sheets of copper, and saw almost every engineering society in the South East showing off their wares. I’d taken my camera along and a sound kit so I did a few interviews just capturing some of the conversations I was having with people. But then we turned a corner and met SMEE. They were all wearing blue work coats – every member of the society has one – and were standing up beside their creations proudly showing them off, answering questions and roping people in to make a pulley tool on a small lathe they had.</i></blockquote>
<h4>
Why did you decide to spend a year following them around, what made their story interesting to you?</h4>
<div>
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/37467868" webkitallowfullscreen="" width="500"></iframe><br /></div>
<div style="text-align: center;">
<i>A short film from the interviews of that day.</i></div>
<blockquote>
<i>On that day, a lot of other people’s creations were incredible displays of ingenuity but you weren’t allowed to touch them. SMEE didn’t really mind about all that. They encouraged everyone to pick stuff up, see how it was made and just generally be inquisitive. I knew then that this was a fascinating bunch of people that I wanted to know more about.</i>
<br />
<div>
<br /></div>
<i>
</i>
<div>
<i><br /></i></div>
<i>I made a short film from the interviews I shot on the day–it’s not a proper documentary, more a sort of film sketch as a tool for exploring some of the conversations I had and grouping them into themes. But that definitely made me want to capture this field in more detail.</i></blockquote>
<h4>
Can you see any parallels between them, and today’s hacker and maker spaces?</h4>
<blockquote>
<i>Definitely. In a way, SMEE are like a hackspace – they have a workshop and a headquarters building where they meet regularly. I guess the difference is that having been around over a hundred years means they have a few more traditions and are more traditionally organised than a hack space – they have a council and a Chairman for example. But the sentiment is the same. They’re a community united by a passion for making things. When I’ve been at SMEE on their workshop evenings, there’s one guy who travels over 2 hours each way just to use the workshop for an hour or two surrounded by fellow members. That fascinates me. I kept asking him why he bothered, why he didn’t use his workshop at home. And all he said was, it’s just not the same.</i><br />
<i><br /></i>
<i>SMEE are really into CNC machining and are interested in 3D printing and other modern techniques too. In the Society film, Norman himself recognises that although there’ll always be people who think if you don’t make everything yourself with files and hammers you’re not doing it right, there’s another set of people who think that 3D printing is the future and CNC machines mean they can be more ambitious with their projects. And I think that’s what I love most about SMEE. They’re not preoccupied with nostalgia, they just happen to have a really rich heritage. What they do is make things.</i></blockquote>
<h4>
Do you think the current films reflect the Society as a whole, or did you focus on a single thread of their story?</h4>
<blockquote>
<i>When I started out, I wanted to make just one film and I began filming and finding people to focus on with that in mind. But as time went on I realised that one film wouldn’t capture the essence of what makes these people who they are. One film would have skipped over little moments like the way Norman sees a ripple pattern in a piece of wood and the way Mike assembles all the little bits of his rail motor. For me, these bits are really important. It’s the power of film to show not just tell. And it reflects the texture of making things, the sounds and the look and the feel of materials. So, if anything, that was the way I approached the collection but it wasn’t really intentional, it just sort of ended up that those were the things that rose to the surface.</i></blockquote>
<h4>
There are four films—an introduction to the Society, the Problem Solver, the Model Engineer and the Woodworker. Will there be more?</h4>
<blockquote>
<i>I did want to make sure I had a sufficient variety of approaches to craft but that wasn’t hard to find at SMEE. I think woodworking, model engineering and experimental engineering reflect a broad range of the types of craft SMEE members engage in but by no means all of them. SMEE have almost 500 members all over the world so I’d love to continue making more.</i></blockquote>
<h4>
What do you think the message of the series is, what story were you trying to tell with the films?</h4>
<blockquote>
<i>If anything, it’s that we’re all makers of things. Sounds a bit cliched but the title The Makers of Things just came from something Norman said. He said that even when he was 14 and had a shed in his parents’ garden he made sawdust. I like the idea that whatever your discipline, your chosen material or intention, you can make stuff. </i></blockquote>
<h4>
What do the members of the Society think of your series? Have they commented?</h4>
</div>
</div>
<blockquote>
<i>Several of them have seen them and have told me they really liked them. I think for them they’ve never really had the chance to see a real outside perspective on what they do. So I’m very pleased they see them as a genuine reflection of their hobby and the characters involved. I’m doing a talk at their next monthly meeting and will be showing the films to the full Society then.</i></blockquote>
<h4>
What other reactions to the series have you had? Have any surprised you?</h4>
<blockquote>
<i>I’m been really chuffed by the reaction online and from family/friends at a screening I had last weekend. Everyone has said they liked the intimacy of the films, that they actually feel they’ve been offered a glimpse into people’s lives. Making stuff is a very personal thing and usually what we see of how other people approach it is a nicely composed photo of their desk all neat and tidy which is not how real life works. People have said they like that the collection feels very real and meaningful. What the people in the film are sharing is a lifetime of making and all the detail, emotion and hard work that comes with that. I’m really pleased people have picked up on that.</i></blockquote>
<h4>
Switching gears then, what equipment did you use to shot the series?</h4>
<blockquote>
<i>The whole series was shot on a Canon EOS 7D with a range of prime lenses, with external radio mics for interviews.</i></blockquote>
<br />Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com19tag:blogger.com,1999:blog-7734900.post-72521196738744162472013-04-02T17:05:00.004+01:002013-04-03T14:59:45.800+01:00Leaving the ivory towerI've been planning to leave academia for some time, but kept on putting it off. Unlike the U.S. where tenure is a thing pursued vigorously by the great and the good, here in the U.K. at least it has long gone to dust. But my job was as permanent as they get, and actually left me a lot of time to do a lot of other things outside it that interested me...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-Knngg9j21do/UVr3GrAYZhI/AAAAAAAATYQ/DB33T_dpuZc/s1600/IMG_8627.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="297" src="http://3.bp.blogspot.com/-Knngg9j21do/UVr3GrAYZhI/AAAAAAAATYQ/DB33T_dpuZc/s400/IMG_8627.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Looking out over the Pacific</i></td></tr>
</tbody></table>
However recently I took a look around and discovered that everything that was getting me up in the morning had nothing to do with my day job, and everything to do with what I was doing outside it. That just isn't any way to live. So, I've just pushed the big red switch. I now have a long rope and will be using it to leave the ivory tower real soon, my last day <a href="http://www.exeter.ac.uk/">here at the University of Exeter</a> is later this week, Thursday the 4th of April.<br />
<br />
I was originally planning to take a couple of months off to look around, mainly because I'm in the fortunate position that I can do that, and such opportunities shouldn't be wasted. However some Tesla-driving individuals said "<i>Yes!"</i> and I've now working on something that's going to swallow my life for the next couple of months.<br />
<br />
However, I'm not complaining, it's just the sort of getting out of bed project that I'm quitting academia to do in the first place. You'll be hearing more about it shortly, just as soon as I can talk about it...<br />
<br />
In the short to medium term I'm planning on <a href="http://babilim.co.uk/">staying freelance</a>, and doing consulting, contracting, writing or anything else that'll pay the bills and keep the wolves from the door. Although I'm not opposed to the idea of joining a (large) company, I've just spent thirteen years working for someone else, it'll be nice to work for myself for a while. Or at least be nearer the top of the tree, as you can generally see the rest of the forest much better from there. That said, it doesn't mean I'm not open to offers; they'd just have to be interesting offers.<br />
<br />
So, while I've got a large number of things that might come off; I'm interested in work. Preferably work of substance, but beggars can't be choosers.<br />
<br />
I've done a number of (some <a href="http://radar.oreilly.com/2011/04/apple-location-tracking.html">quite infamous</a>) things with iOS, and have a lot of experience on the app side of things. I have done a number of things that are now generally being lumped into the<i> "Big Data"</i> camp. While I'm not a Hadoop and NoSQL guy, I've done <a href="http://www.dailyack.com/2011/02/machine-learning-in-real-world.html">some interesting work with machine learning and agent architectures</a>, mostly to do with <a href="http://datasensinglab.com/">distributed sensor networks</a>. I'm a hardware guy, or at least <a href="http://radar.oreilly.com/2011/05/arduino-open-hardware-movement.html">I'm an Arduino guy</a>, and have done <a href="http://datasensinglab.com/">a number of other things</a> to do with that increasingly ubiquitous hardware platform.<br />
<br />
I like playing with mobile platforms, hardware, software, sensors, 3D printers and <a href="http://radar.oreilly.com/2012/09/udid-data-analysis.html">data visualisation</a>. Or preferably all of the above at the same time, a good example of this is the work on the <a href="http://datasensinglab.com/">Data Sensing Lab</a> I've been doing for O'Reilly.<br />
<br />
Basically I'm <a href="http://radar.oreilly.com/2013/01/the-inevitability-of-smart-dust.html">an emerging technology guy</a>. If it's new and a lot of people know nothing about it, I probably know something or am learning about it right now. Then I generally <a href="http://www.oreillynet.com/pub/au/3904#Books">write a book about it</a> and move on to the next emerging technology. I like being on the cutting edge. It's interesting out here. Oh yes, I also helped discover the most distant astronomical object yet found; <a href="http://arxiv.org/abs/0906.1577">a gamma-ray burster at a redshift of 8.2</a>. However I'm not so sure that's a useful skill outside of the ivory tower.<br />
<br />
In summary then; <a href="http://www.oreillynet.com/pub/au/3904#Books">I write</a>, I code, <a href="http://www.youtube.com/watch?v=9yv1_ooM-pI">I speak</a> and am always willing to offer advice on things I know about.<br />
<div>
<br /></div>
<b>Update: </b>It has just been pointed out to me that I <a href="http://www.dailyack.com/2005/11/making-license-plates.html">foresaw my own exit from academia</a> some seven or eight years ago, back when I was still having fun in my day job, <i>"...so what happens when I stop having fun? I'll probably have to sit down and make enough license plates so I don't have to worry about that stuff again."</i>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com46tag:blogger.com,1999:blog-7734900.post-33824394333825584642013-02-23T18:09:00.002+00:002013-04-05T12:59:59.429+01:00Distributed Network DataMy latest book, my first not talking about iOS and writing code for the iPhone and iPad, just went to press. It's called <i>Distributed Network Data</i> and it's hardware hacking for Data Scientists. It's the book of the <a class="g-profile" href="http://plus.google.com/108178010523548991871" target="_blank">+Data Sensing Lab</a> and arrives just in time for this year's <a class="g-profile" href="http://plus.google.com/108539139864366919600" target="_blank">+O'Reilly Strata</a> in Santa Clara, which starts tomorrow.<br />
<br />
<div style="text-align: center;">
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/7jIOFVs4Uns?hl=en_GB&version=3"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/7jIOFVs4Uns?hl=en_GB&version=3" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
</div>
<div style="text-align: center;">
<br /></div>
This book is intended for data scientists who want to learn how to work with external hardware. It assumes some basic computing and programming knowledge, but no real expert knowledge is assumed. From there the book walks you through build your own distributed sensor network to collect, analyse, and visualise real-time data about our environment.<br />
<br />
If you're a data scientist, or a visualisation person, interested in getting started with hardware and collecting your own data, this is the book for you. You can use the code <b>AUTHD</b> to get 40% off print books, 50% on ebooks and videos when you buy the book directly from O'Reilly.<br />
<br />
<iframe frameborder="0" height="150px" scrolling="no" src="http://cdn.oreillystatic.com/widgets/author/70.html" width="500px"></iframe>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com278tag:blogger.com,1999:blog-7734900.post-16799696245028315782013-02-21T00:40:00.001+00:002013-02-21T00:40:44.839+00:00Making money faster than you can type<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">The </span><a data-mce-href="http://www.kickstarter.com/projects/1351910088/3doodler-the-worlds-first-3d-printing-pen" href="http://www.kickstarter.com/projects/1351910088/3doodler-the-worlds-first-3d-printing-pen" style="color: #743399; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;" title="3Doodler">3Doodler</a><span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;"> is a 3D printer, but it's a pen. This takes 3D printing and turns it on its head...</span><br />
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/DQWyhezIze4?version=3&hl=en_GB"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/DQWyhezIze4?version=3&hl=en_GB" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 12px; line-height: 18px;" title="Page 3">
<div style="font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
In fact the 3Doodler rejects quite a lot of what most people would consider necessary for it to be called a 3D printer. There is no three axis control, there is in fact no software, you can't <a data-mce-href="http://www.thingiverse.com" href="http://www.thingiverse.com/" style="color: #743399; line-height: 1.5;">download a design</a> and print an object, it strips 3D printing back to basics.</div>
<div style="font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
What there is, what it allows you to do, is make things. This is the history of printing going in reverse, it's as if Gutenberg's press was invented first, and then somebody came along afterwards and invented the fountain pen.</div>
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/K3QWRI29qCc?hl=en_GB&version=3"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/K3QWRI29qCc?hl=en_GB&version=3" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
<br />
<br />
<div style="font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
While it looks simple they've obviously overcome some serious technological difficulties to get it working. One of the things that's hard to do on 3D printers, at least hard to do well, is unsupported structures.</div>
<div style="font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
As anyone that owns a 3D printer will tell you, the cooling time for the plastic as it leaves the print head is crucial to allow you to print unsupported structures. Too hot and it doesn't work, the structure sags and runs, too cold and it just plain doesn't work at all. From their <a data-mce-href="http://www.youtube.com/watch?v=DQWyhezIze4" href="http://www.youtube.com/watch?v=DQWyhezIze4" style="color: #743399; line-height: 1.5;">videos</a> they seem to have cracked the problem, building a free standing structure seems to be easy and well within the capabilities of the pen.</div>
</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
It also takes 3mm ABS and PLA as its “ink,” the same stuff used by most hobbyist 3D printers. I've got spools of this stuff hanging around my house which I use in <a data-mce-href="https://plus.google.com/u/0/117841261693434574785/posts/WB2dLsnmrYx" href="https://plus.google.com/u/0/117841261693434574785/posts/WB2dLsnmrYx" style="color: #743399; line-height: 1.5;">my own printer</a>. But unlike my printer, which cost just under a thousand dollars, the 3Doodler costs just $75.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
It doesn't have the same capabilities, but that's the difference between a printing press and a pen. It has different capabilities, ones a "normal" 3D printer doesn't have. It's not a cheap alternative, it's a different thing entirely.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
I'm currently watching the 3Doodler climb towards their first million dollars <a data-mce-href="http://www.kickstarter.com/projects/1351910088/3doodler-the-worlds-first-3d-printing-pen" href="http://www.kickstarter.com/projects/1351910088/3doodler-the-worlds-first-3d-printing-pen" style="color: #743399; line-height: 1.5;">on Kickstarter</a>, and I when I say their first million I mean that, they have over 30 days to go on their campaign which has today has gone viral and made them the best part of that million. This is the next <a data-mce-href="http://www.kickstarter.com/projects/597507018/pebble-e-paper-watch-for-iphone-and-android" href="http://www.kickstarter.com/projects/597507018/pebble-e-paper-watch-for-iphone-and-android" style="color: #743399; line-height: 1.5;">Pebble</a>. The next Kickstarter success story.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
They've tapped into a previously untappable market; people that wanted a 3D printer but couldn't afford one, and people that see the obvious potential of a fountain pen over a printing press, for both art and engineering.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
The guys behind the <a data-mce-href="http://www.the3doodler.com" href="http://www.the3doodler.com/" style="color: #743399; line-height: 1.5;">3Doodler</a> made $60,000 dollars while I wrote this post, my hat is off to them. Because it's not often someone comes up with an idea this good.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
I'm going to be writing a series of posts on hardware startups <a href="http://radar.oreilly.com/aallan">for the Radar</a> over the course of the next few months, and rest assured I'll come back to the 3Doodler. But not until they can type faster than they can make money.</div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com266tag:blogger.com,1999:blog-7734900.post-70829067531791892982013-02-13T12:21:00.003+00:002013-02-13T12:21:54.339+00:00The black rectangle won't last as long as the beige box<!--?xml version="1.0" encoding="UTF-8" standalone="no"?-->
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="background-color: white; line-height: 18px;">It looks like putting </span><a href="https://plus.google.com/s/%23Linux" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Linux</a><span style="background-color: white; line-height: 18px;"> on </span><a href="https://plus.google.com/s/%23Microsoft" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Microsoft</a><span style="background-color: white; line-height: 18px;">'s new </span><a href="https://plus.google.com/s/%23Surface" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Surface</a><span style="background-color: white; line-height: 18px;"> is </span><a href="http://paritynews.com/hardware/tablets/item/531-linux-on-microsoft-surface-what-are-the-odds">going to be an up hill struggle</a><span style="background-color: white; line-height: 18px;">. I was actually expecting that...</span></span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br style="background-color: white; line-height: 18px;" /></span></div>
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">The era of the commodity beige box is coming to an end, and the days of the general purpose computer are almost over. Most people never needed or wanted a general purpose computer, and they're going to be happy with more limited devices optimised for a single, or a few, purposes. So long as those devices just work.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br style="background-color: white; line-height: 18px;" /></span></div>
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">As a scientist I've benefited from being able to take mass produced PCs and be able to put them on desks very cheaply. The amount of compute power we've had access to as a result meant that money that would otherwise have been spent on expensive high end workstations could be spent elsewhere.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br style="background-color: white; line-height: 18px;" /></span></div>
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">Those of us that need general purpose computing; designers, developers, scientists, are going to have to go out and buy increasingly expensive niche machines, effectively old-fashioned workstations. High end computing platforms that the general population just don't need on their desk or in their pocket.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br style="background-color: white; line-height: 18px;" /></span></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="background-color: white; line-height: 18px;">The fact you can't install </span><a href="https://plus.google.com/s/%23Linux" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Linux</a><span style="background-color: white; line-height: 18px;"> on the new </span><a href="https://plus.google.com/s/%23Surface" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Surface</a><span style="background-color: white; line-height: 18px;"> is just the start of what is going to be an increasingly obvious trend. It's just a symptom. The things that are open and the things that are closed are changing. Time to wake up and realise that. Being able to install </span><a href="https://plus.google.com/s/%23Linux" style="background-color: white; color: #3366cc; cursor: pointer; line-height: 18px; text-decoration: none;">#Linux</a><span style="background-color: white; line-height: 18px;"> on your PC isn't important any more.</span></span><br />
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; line-height: 18px;"><br /></span></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="background-color: white; line-height: 18px;">I think a lot of the web and mobile people are making the same mistake today that </span><a href="http://techcrunch.com/2012/12/31/nokias-long-drawn-out-decline/">Nokia made five years ago</a><span style="background-color: white; line-height: 18px;">, Nokia was all about the hardware and wasn't watching the software hard enough...</span></span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br style="background-color: white; line-height: 18px;" /></span></div>
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">Today people are all about the software and aren't watching the hardware hard enough. Today's mobile phone, the black rectangle with, at most, a single button is a transition device. Don't get too comfortable with it, and don't stop thinking about innovation. Because the black rectangle won't last as long as the beige box.</span></div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com38tag:blogger.com,1999:blog-7734900.post-88679341928284466432013-02-03T19:32:00.003+00:002013-02-03T19:32:53.121+00:00Predicting a SingularityI think a lot about the future, and because of that I've gained somewhat of a reputation for making good predictions. This is a characteristic I share with prophets, messiahs and other ne'er-do-wells. I'm not entirely sure what to think about that.<br />
<br />
However one of the problems with making predictions about the future, the main problem, is that it's actually not that hard to predict what'll happen next year. Although for some reason this doesn't really seem to help many of the major analysts whose job it is to make such forecasts. Conversely it's also not that hard to make a prediction for the far future, as <i>"…any sufficiently advanced technology is indistinguishable from magic."<sup>1</sup></i> Why is this a problem? Well if it's easy to make those predictions, making predictions in the sweet spot, both far enough ahead to be ahead to get you ahead of your competition, and close enough that you'll still be around to do something about it is actually almost impossible.<br />
<br />
But how far in the future is the sweet spot? The strange thing is that this actually changes, a century ago it was twenty years, or even thirty, but twenty or thirty years ago it was just ten years. Today it's probably five, or less. The rate of technological progress is accelerating, and with it the amount of change we'll experience during our lives is also changing. The time it takes for new technologies to emerge, become mainstream, become dated and then obsolete is falling almost exponentially.<br />
<br />
For someone like me, whose career more or less relies on being on, and being seen to be on, the bleeding edge, this is painfully evident. If I'm asked <i>"Have you heard about…?" </i>and I have to answer <i>"No?"</i> you'll generally see a look of pain cross my face, something sort of like constipation, don't worry, it's just my career flashing before my eyes...<br />
<br />
At this point having angered both business analysts and science fiction writers I'm going to make a small admission, both professions have the right of it because <i>"...the future is already here — it's just not very evenly distributed."<sup>2</sup></i><br />
<br />
While the pace of change has accelerated, it's hard to see how that can be sustained as the size of the install base of existing legacy technology widens. So far we seem to have sustained that pace of change by trickle down economics, the older technology spreads out, and newer technology is dropped in, eagerly seized by early adopters like myself willing to pay the premium, and experience the inevitable problems that come with all new technology, at least until the bugs are shaken out.<br />
<br />
Despite that I'm forced to point out that if you extrapolate the current rate of technological progress the view that some sort of technological singularity must almost be inevitable is hard to argue against. Unless of course there is some sort of major catastrophe, something to set us back.<br />
<br />
Major catastrophes that could knock us back aren't hard to spot: a global pandemic, climate change, ecological disaster, super volcanoes, mega-tsunami, overpopulation, asteroid impact, a nearby supernovae and of course worldwide thermonuclear war are all favourites. The threat of some of these seems to be fading, but some seem more likely today than ever. There are others, many others, too numerous and depressing to list here.<br />
<br />
They're also wildcards, because sometimes the things that should set us back push us forward. It's certainly arguable that two major world wars, so close together, were a major causal factor in the acceleration of the pace of change that is part of our lives today.<br />
<br />
Depending on the current news cycle I can swing violently; between a horribly over optimistic view of the future and the inevitability of the rise of trans-humanism, and a view bleak in pessimism, in the inevitability of the abandonment of technology and a slow slide towards narrowing horizons and the eventual extinction of the human race. Doomed as a species that turned its back on space and by having its world view limited to just that, a single world with all the disasters and catastrophes that can result.<br />
<br />
Despite this, I'll continue to try and make predictions in the sweet spot, it's fun to be proved right, and sometimes even more fun to be proved wrong.<br />
<br />
<i><sup>1</sup> Arthur C. Clarke</i><br />
<i><sup>2</sup> William Gibson</i>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com118tag:blogger.com,1999:blog-7734900.post-49514240557766572442013-02-02T20:33:00.000+00:002013-02-02T20:33:10.244+00:00You don't have to be awesome all the time...<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">In her post <a href="http://www.zephoria.org/thoughts/archives/2013/02/02/mourning-and-public-ness.html">talking about the public-ness of mourning</a> after the <a href="http://www.dailyack.com/2013/01/goodbye-aaron.html">death of Aaron Swartz</a></span><i style="background-color: white; color: #5e5e5e; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">, </i><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">Danah Boyd writes </span><i style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">"...we’ve created communities connected around ideas and actions, relishing individualistic productivity for collective good. But we haven’t created openings for people to be weak and voice their struggles and demons."</i><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">Geek culture is, at least in theory, a meritocracy, and you are measured by your accomplishments. But that means the best of us, those whose work is held up as shining examples, suffer from <a href="http://en.wikipedia.org/wiki/Impostor_syndrome">Impostor Syndrome</a>. Sometimes cripplingly so, even when they are accomplishing awesome things. Because awesome things sometimes look a lot less awesome from the inside, when you know the limitations, flaws and problems with what you've built and shared with the community.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">But worse than that, it means when you have your moment of weakness (and we all do), and <a href="http://books.google.com/books?id=7papZR4oVssC&lpg=PA173&dq=%22Now%20that%20I%20am%20burned%20out%20and%20I'll%20never%20accomplish%20anything%22&pg=PA173#v=onepage&q=%22Now%20that%20I%20am%20burned%20out%20and%20I'll%20never%20accomplish%20anything%22&f=false">for a while cannot contribute</a>, cannot accomplish the day-to-day awesomeness that qualifies you as a member of good standing of the community, things can look very bleak. Because we expect the most from those of us that deliver the most, and even the great and the good can fall sometimes, and need support.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">We've built a culture where it's hard to acknowledge that you don't know something, because knowing things is intricately linked with the doing of awesome things </span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">—</span><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"> which in turn is linked to our stature with our peers.</span><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">For someone like me, whose career more or less relies on being on, and being seen to be on, the bleeding edge, this is painfully evident. If I'm asked </span><i style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">"Have you heard about…?"</i><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"> and I have to answer </span><i style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">"No?"</i><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"> you'll generally see a look of pain cross my face, something sort of like constipation, don't worry, it's just my career flashing before my eyes…</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">I have no solutions to offer, only the sure and certain knowledge, which I give freely to other geeks, that you are not alone. That the great and the good amongst us suffer as well. That it's okay to be weak and not know the answer to a question. That it's okay to rest and take from others for a while. We'll still be here when you get back, and we'll still remember how awesome you are. You don't have to live your life on Internet time.</span>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com35tag:blogger.com,1999:blog-7734900.post-4416375787864844562013-01-12T10:26:00.000+00:002013-01-16T10:35:01.490+00:00Goodbye Aaron<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">I heard this morning that </span><i style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"><a href="http://en.wikipedia.org/wiki/Aaron_Swartz">Aaron Swartz</a></i><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"> has committed suicide. He was just twenty six. That's far too young by anyone's measure.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-zRxXNX2_AiM/UPaAoa4rRdI/AAAAAAAAP3o/ADcfo5sH8uA/s1600/Aaron_Swartz_profile.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="297" src="http://1.bp.blogspot.com/-zRxXNX2_AiM/UPaAoa4rRdI/AAAAAAAAP3o/ADcfo5sH8uA/s400/Aaron_Swartz_profile.jpg" width="400" /></a></div>
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">It's unclear how much the pressures of the unreasonably harsh federal prosecution for the JSTOR incident might have weighed on him, because it's been clear that he was depressed for some years. Like many of us that suffer from bouts of depression he had good weeks, and bad weeks. But the legal mess he was in can hardly have been a light weight to bear.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">We've had several well known people in the community commit suicide over the last couple of years, and it's jarring. From the outside they look like the best of us, the brightest, sometimes with the most to lose. From the inside it can look much bleaker.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">People in our community grew up geeks, many grew up friendless and carry that burden into adulthood. They have real trouble reaching out when they need help; to the friends they're not sure they really have, to the family they often regard as having not been there for them when they were at school. As a result the community is littered with people that suffer depression, that struggle every day with it, and with Impostor Syndrome. No matter how accomplished people look on the outside, and despite past records that should make those accomplishments as evident to them as it is to the rest of us, they suffer. Often in silence. </span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">I didn't know Aaron well, we had exchanged a few words on a couple of occasions, but I should have had a chance to fix that. He was twenty six and he was at the start of things, not the end.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">If you feel like you can't go on, if you feel like it's too much to bear the weight of your life alone. Please, don't do this, please reach out to your friends, your family, to strangers if you must. If you can't face your friends with the news that you hate your life. Because there is always someone that's going to miss you. Always.</span><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><br style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;" /><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">Goodbye Aaron.</span>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com14tag:blogger.com,1999:blog-7734900.post-81301636863547539442013-01-08T14:22:00.000+00:002013-01-08T14:22:49.068+00:00The inevitability of smart dust<div style="text-align: center;">
This post was first published on the <a href="http://radar.oreilly.com/2013/01/the-inevitability-of-smart-dust.html">O'Reilly Radar</a>.</div>
<br />
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
I've <a data-mce-href="http://radar.oreilly.com/2011/05/next-big-thing-web-mobile-data-ubiquitious-computing.html" href="http://radar.oreilly.com/2011/05/next-big-thing-web-mobile-data-ubiquitious-computing.html" style="color: #743399; line-height: 1.5;">put forward my opinion that desktop computing is dead</a> on more than one occasion, and been soundly put in my place as a result almost every time. <em style="border: none; line-height: 1.5;">"Of course desktop computing isn't dead — look at the analogy you're drawing between the so called death of the mainframe and the death of the desktop. Mainframes aren't dead, there are still plenty of them around!"</em></div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
Well, yes, that's arguable. But most people, everyday people, don't know that. It doesn't matter if the paradigm survives if it's not culturally acknowledged. Mainframe computing lives on, buried behind the scenes, backstage. As a platform it performs well, in its own niche. No doubt desktop computing is destined to live on, but similarly behind the scenes, and it's already fading into the background.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
The desktop will increasingly belong to niche users. Developers need them, at least for now and for the foreseeable future. But despite the prevalent view in Silicon Valley, the world does not consist of developers. Designers need screen real estate, but buttons and the entire desktop paradigm are a hack; I can foresee the day when the computing designers use will not even vaguely resemble today's desktop machines.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
For the rest of the world? Computing will almost inevitably diffuse out into our environment. Today's mobile devices are transition devices, artifacts of our stage of technology progress. They too will eventually fade into their own niche. Replacement technologies, or rather user interfaces, like Google's <a data-mce-href="https://plus.google.com/+projectglass/posts" href="https://plus.google.com/+projectglass/posts" style="color: #743399; line-height: 1.5;">Project Glass</a> are already on the horizon, and that's just the beginning.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
People never wanted computers; they wanted what computers could do for them. Almost inevitably the amount computers can do for us on their own, behind our backs, is increasing. But to do that, they need data, and to get data they need sensors. So the diffusion of general purpose computing out into our environment is inevitable.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
Everyday objects are already becoming smarter. But in 10 years' time, every piece of clothing you own, every piece of jewelry, and every thing you carry with you will be measuring, weighing and calculating. In 10 years, the world — your world — will be full of sensors.</div>
<center>
<iframe allowfullscreen="allowfullscreen" frameborder="0" height="253" src="http://www.youtube.com/embed/9yv1_ooM-pI" width="450"></iframe></center>
<div class="separator" style="clear: both; text-align: center;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">The sensors you carry with you may well generate more data every second, both for you and about you, than previous generations did about themselves during the course of their entire lives. We will be surrounded by a cloud of data. While the phrase "data exhaust" has already entered the lexicon, we're still essentially at the </span><a data-mce-href="http://bit.ly/senselab" href="http://bit.ly/senselab" style="color: #743399; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">banging-the-rocks-together stage</a><span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">. You haven't seen anything yet ...</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;"><br /></span></div>
<center>
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/n4tGWCiaFNs?hl=en_GB&version=3"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/n4tGWCiaFNs?hl=en_GB&version=3" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object></center>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">The end point of this evolution is already clear: it's called </span><a data-mce-href="http://en.wikipedia.org/wiki/Smartdust" href="http://en.wikipedia.org/wiki/Smartdust" style="color: #743399; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">smart dust</a><span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">. General purpose computing, sensors, and wireless networking, all bundled up in millimeter-scale sensor motes drifting in the air currents, flecks of computing power, settling on your skin, ingested, will be </span><a data-mce-href="http://bits.blogs.nytimes.com/2012/09/07/big-data-in-your-blood/" href="http://bits.blogs.nytimes.com/2012/09/07/big-data-in-your-blood/" style="color: #743399; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">monitoring you inside and out</a><span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;">, sensing and reporting — both for you and about you.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5;"><br /></span></div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
Almost inevitably the amount of data that this sort of technology will generate will vastly exceed anything that can be filtered, and distilled, into a remote database. The phrase "data exhaust" will no longer be a figure of speech; it'll be a literal statement. Your data will exist in a cloud, a halo of devices, tasked to provide you with sensor and computing support as you walk along, calculating constantly, consulting with each other, predicting, anticipating your needs. You'll be surrounded by a web of distributed sensors and computing.</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
Makes desktop computing look sort of dull, doesn't it?</div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com21tag:blogger.com,1999:blog-7734900.post-5090719371210370242013-01-06T12:27:00.000+00:002013-01-06T12:29:33.443+00:00The fifth horseman never gets invited to the good parties<div style="text-align: center;">
This article was originally posted on <a href="https://plus.google.com/u/0/117841261693434574785/posts/Y8i86jmouLm">Google+</a>.</div>
<br />
Yesterday <a class="g-profile" href="http://plus.google.com/107753428759636856492" target="_blank">+MG Siegler</a> argued on <a class="g-profile" href="http://plus.google.com/103037366582313115962" target="_blank">+TechCrunch</a> that Samsung <a href="http://techcrunch.com/2013/01/05/the-fifth-horsemen-of-tech-samsung/">is the fifth horseman of technology</a>, filling in for the ailing <a href="http://microsoft.com/">Microsoft</a>, when the four horsemen: <a href="http://amazon.com/">Amazon</a>, <a href="http://apple.com/">Apple</a>, <a href="http://facebook.com/">Facebook</a> and <a href="http://google.com/">Google</a> go riding.<br />
<br />
I have to disagree with the underlying assumptions. We're at yet another tipping point in technology. A few years ago we moved from the beige box to the black rectangle, but the black rectangle won't be with us for as long as the beige box.<br />
<br />
That black rectangle, the ubiquitous form factor of today's smart phone, is a transition device and it's going to disappear quickly as the speed of technological change is accelerating rapidly. Of the four horsemen only Google seems to be working on alternatives with <a class="g-profile" href="http://plus.google.com/111626127367496192147" target="_blank">+Project Glass</a>. It's possible the others, including Samsung, have working hardware, but the successor to today's smart phone is going to be all about context and user interaction.<br />
<br />
I've stood up in front of audiences before and argued that our smart phones have our lives on them, the next generation of mobile technology is going to stand between us and our lives and add context. It's hard to do that without a lot of information about the user.<br />
<br />
It's also going to be a big leap for the horsemen to make. Despite getting into hardware recently Amazon is about selling content, Facebook has never done hardware and I don't think have this sort of paradigm shift in their corporate bones, Apple has, but without Steve Jobs I don't think they'll have the guts to kill the iPhone and innovate. Samsung, the fifth horsemen that never gets invited to the good parties, is a box shifter. They know hardware, but they don't know design, and they don't know anything about their end users. Their customers are other companies, like Amazon, not you and me, the eventual consumers.<br />
<br />
So out of all of them Google seems to be the only one positioned to move forward, and it'll be a big leap for them even so. The developer release of <a class="g-profile" href="http://plus.google.com/111626127367496192147" target="_blank">+Project Glass</a> later this year is going to be crucial. If I had the money to lose making a wager, I'd wager that it'll be some startup you or I haven't heard of yet that makes the leap to the next ubiquitous form factor.<br />
<br />
Either way, it's going to be an interesting year...Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com10tag:blogger.com,1999:blog-7734900.post-622192769935755992012-12-28T13:39:00.002+00:002012-12-28T13:48:28.733+00:00The rise of the personal space program<div style="text-align: center;">
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">This article was originally posted on <a href="https://plus.google.com/117841261693434574785/posts/PixeBPqq8uM">Google+</a>.</span></div>
<div style="text-align: center;">
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;"><br /></span></div>
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">Just over a year ago now, the first ever project I backed on <a href="http://kickstarter.com/">Kickstarter</a> </span><span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">was <a href="http://bit.ly/kicksat">Kicksat</a></span><span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">. A project to put a swarm of small nano-satellites in orbit. The size of a couple of postage stamps each satellite has solar cells, a radio transceiver, and a micro-controller along with memory and sensors.</span><br />
<br style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;" />
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">Today in the mail my souvenir satellite arrived, it's just 3.5 cm square. It's an engineering prototype, presumably one that failed verification, and it doesn't have it's solar cells attached, but otherwise its just like the ones that'll be flown into orbit.</span><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-qn_SoNsguH0/UN2glgTVRZI/AAAAAAAAPHU/_Z0W_Otek6U/s1600/IMG_8075.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://4.bp.blogspot.com/-qn_SoNsguH0/UN2glgTVRZI/AAAAAAAAPHU/_Z0W_Otek6U/s400/IMG_8075.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>An engineering sample of a flight ready Sprite nano-satellite.</i></td></tr>
</tbody></table>
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">At it's heart is a </span><span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;"><a class="g-profile" href="http://plus.google.com/104292131839044508100" target="_blank">+Texas Instruments</a> <a href="http://www.ti.com/product/cc430f5137">CC430F5137</a>. It's a whole system on a chip based around an <a href="http://www.ti.com/lsds/ti/microcontroller/16-bit_msp430/overview.page?DCMP=MCU_other&HQS=msp430">MSP430</a> CPU, a 16-bit <a href="http://en.wikipedia.org/wiki/Reduced_instruction_set_computing">RISC</a> ulta-low power micro-controller, along with an onboard RF transceiver operating in the sub-1GHz bands, a real-time clock and an integrated temperature sensor.</span><br />
<br style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;" />
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">As well as the big solder pads at the top of the board for the solar cell there are a number of unpopulated pads on the board, including space for some through hole components, so there is certainly room for more sensors to be added on the board itself, and the MSP430 has the capacity to handle them. </span><br />
<br style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;" />
<span style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">However the satellite is going to be entirely reliant on the solar cell for power, there is no battery back up on the board, and the satellite will only be able to operate on the Sun-ward side of its orbit. That means power is the limiting factor. The CC4305137 has good brown-out reset capability, probably one of the reasons it was chosen, however you have to wonder how much more can be crammed onto the board and still reliably operate. </span><br />
<br />
<div style="background-color: white; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">
While it's might not seem much beyond a shrunken down <a href="http://en.wikipedia.org/wiki/Sputnik_1">Sputnik</a> at that point, the <a href="http://bit.ly/kicksat">Kicksat</a> is ground breaking. In just fifty-five years we've advance from the point where it takes the might of one of the world's only super powers to put something like this into orbit, to the point where several hundred of them can be put into orbit by a graduate student with some enthusiastic backers.<br />
<br />
Despite the pessimism I often express at the way the space programme is going, this is something that gives me a lot of hope that we aren't going the wrong way. That we aren't starting a march towards the abandonment of technology and a slow fall towards an age of declining possibilities and narrowing horizons.<br />
<br /></div>
<div style="background-color: white; text-align: left;">
<a href="http://bit.ly/kicksat" style="background-color: transparent; font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">Kicksat</a><span style="font-family: Arial, sans-serif; font-size: 13px; line-height: 18px;">, the <a href="http://en.wikipedia.org/wiki/CubeSat">CubeSat</a> designed to carry hundreds of these small Sprite nano-satellites aboard, is now on track for launch in the autumn of 2013 onboard the <a href="http://en.wikipedia.org/wiki/SpaceX_CRS-3">CRS-3</a>/ELaNa-5 mission. This will be the third Commercial Resupply Service (CRS) flight to deliver supplies to the <a href="http://www.nasa.gov/mission_pages/station/main/index.html">International Space Station</a> (ISS) aboard a <a class="g-profile" href="http://plus.google.com/104512038508075599339" target="_blank">+SpaceX</a> <a href="http://en.wikipedia.org/wiki/Falcon_9">Falcon 9</a> rocket. KickSat, along with 5 other CubeSats, will be hitching a ride as a secondary payload thanks to <a class="g-profile" href="http://plus.google.com/102371865054310418159" target="_blank">+NASA</a>'s <a href="http://www.nasa.gov/offices/education/centers/kennedy/technology/elana_feature.html">ELaNa</a> program.</span></div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com15tag:blogger.com,1999:blog-7734900.post-83230639101282022162012-11-11T19:01:00.001+00:002012-11-23T11:57:07.809+00:00Teardown of Wireless Sensor TagsThe <a href="http://mytaglist.com/">Wireless Sensor Tag</a> is a smart tag system by <a href="http://www.caogadgets.com/">CAO Gadgets</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-MXhmuzPb3UQ/UJ-kh3G59GI/AAAAAAAAMYE/HTX9532uTc8/s1600/redtag1_held_in_hand_500h.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="252" src="http://3.bp.blogspot.com/-MXhmuzPb3UQ/UJ-kh3G59GI/AAAAAAAAMYE/HTX9532uTc8/s400/redtag1_held_in_hand_500h.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The Wireless Sensor Tag<br />(Credit: CAO Gadgets)</i></td></tr>
</tbody></table>
The system is based around an <i>Ethernet Tag Manager</i>, a small box that connects directly to your home router and manages the all the associated tags. Basically it acts as a bridge between the wireless tags themselves and the Cloud backend which manages the tags.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://2.bp.blogspot.com/-rH9O6uI7vmo/UJ-k2S3czPI/AAAAAAAAMYM/93OCba4BMbE/s1600/tagmanager_spec.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="217" src="http://2.bp.blogspot.com/-rH9O6uI7vmo/UJ-k2S3czPI/AAAAAAAAMYM/93OCba4BMbE/s400/tagmanager_spec.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;"><i>The Tag Manager<br />(Credit: CAO Gadgets)</i></td></tr>
</tbody></table>
Each tag has motion and temperature sensors and can be configured to notify you via tweet, email, push notification, or via a URL callback (either to an Internet facing host or directly to an internal IP on your home network) when the tag is moved, or when the temperature goes out of a user defined range, or if the tag itself is moved out of range of the tag manager. Each tag also has a piezo electric buzzer and an LED which can be triggered using the same ruleset.<br />
<br />
<b>Ordering</b><br />
<br />
I actually ordered my system back in August, only to be told that the sensor tags had not been CE certified, and they weren't quite prepared to do that just yet. I could either cancel my order, or wait until they had enough orders to that it would make <i>"economic sense"</i> for them to obtain CE marking at which point they'd ship me my order.<br />
<br />
I have to admit I wasn't that impressed by that. If you're offering something for sale internationally then you should have the items in stock and you should be able to ship it out of the country. If they weren't prepared to ship into the EU, they shouldn't have processed my order, or billed my credit card.<br />
<br />
They also seem to be having supply chain problems as their current stock is sold out and their <a href="http://wirelesstags.myshopify.com/collections/all">store</a> has a note on the front page saying that the lead time for a new order is currently one to one and half months between order placement to fulfilment.<br />
<br />
However it looked like someone had put a lot of thought into this problem space, and despite the number of similar systems that are starting to appear, the system should be a neat solution. So I hung on, and I'm glad I did, as I I'm actually quite impressed with the build quality of the hardware and how it hangs together as a system...<br />
<b><br /></b>
<b>Unboxing</b><br />
<br />
My system arrived in a plain USPS box. Inside was the tag manager, an ethernet cable, a USB power adapter and cable, ten tags with velcro strips, and somewhat oddly, seven spare batteries.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-Vj8_w_6dR2A/UJ7faBYYcNI/AAAAAAAAMWM/8hnYcmw5IAo/s1600/IMG_7543.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://4.bp.blogspot.com/-Vj8_w_6dR2A/UJ7faBYYcNI/AAAAAAAAMWM/8hnYcmw5IAo/s400/IMG_7543.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The Wireless Sensor Tag System</i></td></tr>
</tbody></table>
<a href="http://wirelesstag.net/webapp.html">Setting up</a> the system is a fairly simple procedure. Plug the tag manager into your router and let it grab a DHCP address and announce itself to the <a href="http://mytaglist.com/eth/">mytaglist.com server</a>. Theoretically the tags included in the tag manager in your shipment should come pre-associated, but at least for me this didn't seem to be the case, but that said associating them is no big deal. Pull the battery tab, and if you have a flashing LED that means the tag is unassociated, go to the web app and click on the button to add a new tag. The tag then shows up in the list on the web backend and you can go ahead and configure the triggers for that tag.<br />
<br />
<b>The Software</b><br />
<div>
<div>
<br /></div>
<div>
Unfortunately despite initial promise, I'm not finding the system amazingly reliable as yet. Tags that are within a few feet of the tag manager are being periodically being reported as <i>"Out of Range"</i> by the web interface. However it's possible that the hardware is fine and I'm just not understanding the software.<br />
<br />
<i><b>Update (12 Nov):</b> Just got an email from the manufacturer saying that they've "fixed some bugs on the server" and these spurious "Out of Range" notifications shouldn't happen any more. It's a bit soon to tell one way or the other, but certainly I'm not seeing them at this point, so this might well be fixed now.</i></div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-JtD2JT_ayRM/UJ-zxbp47mI/AAAAAAAAMYg/eNMW6b1lqVc/s1600/Screen+Shot+2012-11-11+at+14.07.40.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="248" src="http://1.bp.blogspot.com/-JtD2JT_ayRM/UJ-zxbp47mI/AAAAAAAAMYg/eNMW6b1lqVc/s400/Screen+Shot+2012-11-11+at+14.07.40.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The web application used to manage the tags on your system</i></td></tr>
</tbody></table>
<div>
The web application is confusing, the interface basically looks like someone has exposed the backend database schema in software without much thought as to how the user is going to interact with it. At the moment the interesting hardware is being let down badly by the back end software, there should be preset options, e.g. <i>"this tag is attached to a door"</i>, <i>"this tag is attached to a moveable object"</i>, that would auto-configure the tag to common presets.</div>
<div>
<br /></div>
<div>
As it is there are numerous settings to go through for each and every tag, and most of these aren't well explained, and the buttons to set the options are not self explanatory or just flat out confusing. </div>
<div>
<br /></div>
<div>
I'm sure it makes sense to the programmer that built it, as they understand exactly how the system works and how the components interact, to the rest of us, at least initially, it's confusing. </div>
<div>
<br /></div>
<div>
I've been playing with the system for about an hour now and I still can't figure out how to get a tag to start bleeping when moved, and then keep on bleeping until manually reset. It's something you'd commonly want to do for a tag that's attached to something that might be stolen, and the system should be able to do it, but I can't figure it out. It's probably obvious once you know how, but...</div>
<div>
<br /></div>
<div>
They need to get a good <a href="http://en.wikipedia.org/wiki/User_experience">UX</a> person in, along with a designer, to overhaul the backend. That could make a big difference to the software and its usability for the average person. The system itself is really elegant, and easy to get up and working, the software to configure the tags is letting it down. </div>
<div>
<br /></div>
<div>
<strike>Another frustration for me is that the <a href="http://wirelesstag.net/iosapp.html">iOS app</a> that allows you to monitor the tags, and receive push notifications to your phone when one of your triggers is tripped, is <b>only available in the US App Store</b>. The ability to get push notifications was one of the major reasons for me to buy this system, so I'm hoping this is going to be resolved quickly. Again, if you're going to ship outside the US you need to make sure your system is ready to go there.</strike><br />
<br />
<i><b>Update (12 Nov):</b> The iOS app is now available <a href="https://itunes.apple.com/gb/app/wireless-tag-list/id508973799?mt=8">in the UK store</a>, which resolves one of my main problems with the system. Although unfortunately the software UX problems extend to the iOS application. It's comprehensive, but not easy to use.</i><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><br />
<a href="http://4.bp.blogspot.com/-0HoBxJlftMs/UKDNdJcVWpI/AAAAAAAAMeQ/EH0hh_IrBUE/s1600/iphone5.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="400" src="http://4.bp.blogspot.com/-0HoBxJlftMs/UKDNdJcVWpI/AAAAAAAAMeQ/EH0hh_IrBUE/s400/iphone5.png" width="231" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The iOS Tag Manager application running on my iPhone 5</i></td></tr>
</tbody></table>
</div>
<div>
<b>The Hardware</b></div>
<div>
<br /></div>
</div>
<div>
There is actually very little information about how the system hardware works, and I'm not in favour of <i>"just magic"</i>, so I wanted to take a closer look at the board to try and figure out what's going on under the hood.<br />
<br />
The rubberised enclosures are more-or-less indestructible and amazingly stretchy so they're fairly easy to take off, which is a good thing as you'll need to do that before pulling out the battery tab to activate the tag as they tend to break off if you try and do this with the enclosure still on...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://1.bp.blogspot.com/-e2wF8L91ydc/UJ7UEGjTh7I/AAAAAAAAMV0/heSzMqydPEE/s1600/IMG_7544.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://1.bp.blogspot.com/-e2wF8L91ydc/UJ7UEGjTh7I/AAAAAAAAMV0/heSzMqydPEE/s400/IMG_7544.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;"><i>Wireless Sensor Tag</i></td></tr>
</tbody></table>
The tag is powered by a single <a href="http://en.wikipedia.org/wiki/CR2032_battery">CR2032</a> button cell Lithium battery which they're claiming will last anywhere between two months and seven years depending on the sensitivity and polling interval you choose for the tag.<br />
<div>
<br /></div>
<div>
The onboard processor is a Microchip <a href="http://ww1.microchip.com/downloads/en/DeviceDoc/41402A.pdf">PIC16LF720</a> micro-controller, an interesting choice that draws about a fifth of the current than the Amtel <a href="http://www.atmel.com/Images/doc2549.pdf">ATmega</a> micro-controllers used in the familiar Arduino boards. Wireless operations for the tags is provided by the Microchip <a href="http://ww1.microchip.com/downloads/en/DeviceDoc/70590C.pdf">MRF 49XA</a> radio, which operates in the sub-GHz MHz <a href="http://en.wikipedia.org/wiki/ISM_band">ISM</a> bands. While this can be changed, the tags ship and by default operate at 436 MHz. Unlike the US this is <a href="http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CCIQFjAA&url=http%3A%2F%2Fwww.ofcom.org.uk%2Fstatic%2Farchive%2Fra%2Ftopics%2Fresearch%2Frrac%2Fmembers%2Frrac01_19.doc&ei=buifULCqJ6rE0QXC4IHQCA&usg=AFQjCNHX3DTx1xBVtPuNicUzDwe2YvTQWQ&sig2=SnddV7_ewljHsV0eSpkbiw">not an ISM band</a> here in the UK, and although it is part of the amateur band, a license to operate is required and the primary user of the band is the <a href="http://www.mod.uk/">MOD</a>.</div>
<div>
<br /></div>
<div>
The choice of the sub-GHz ISM band is an interesting one, most competing products on the market use the 13MHz <a href="http://en.wikipedia.org/wiki/Radio-frequency_identification">RFID</a> bands, or more commonly the 2.4GZ band used by both <a href="http://en.wikipedia.org/wiki/Wi-Fi">Wi-Fi</a>, <a href="http://en.wikipedia.org/wiki/ZigBee">Zigbee</a> and <a href="http://en.wikipedia.org/wiki/Bluetooth_low_energy">Bluetooth LE</a> devices. I'm guessing that they went with the sub-GHz bands to keep power usage to a minimum and extend the battery life of the tags themselves and provide a good range.</div>
<div>
<br /></div>
<div>
Motion sensing is provided by an Honeywell <a href="http://www51.honeywell.com/aero/common/documents/myaerospacecatalog-documents/Defense_Brochures-documents/HMC5883L_3-Axis_Digital_Compass_IC.pdf">HMC5883L</a> sensor, a three-axis magnetometer, again an interesting choice as most of the competitors are using linear accelerometers. I'm presuming they used a magnetometer to get good angular measurements, which is a perfect fit for one of the main uses cases for the tags; checking whether doors are open or closed. </div>
<div>
<br /></div>
<div>
Finally I'm guessing temperature measurements are provided by a fairly anonymous chip marked with <i>"AXWB." </i>While that's just a guess, the word <i>"TEMP"</i> is the silkscreen legend next to it, so it's probably a decent one. I haven't been able to track down the data sheet for this chip or the manufacturer, although the specifications page gives an operating range of -40°C to 85°C with an typical accuracy of <span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">±</span>1°C and a -2 to +4°C maximum deviation, with a sensor quantisation of around 1°C. </div>
<div>
<br /></div>
</div>
<div>
<b>Developer SDK</b></div>
<div>
<br /></div>
<div>
The system isn't <i>"open hardware"</i> but that's just fine, not everything has to be open, people have to eat and keep a roof over their head and that takes money. However I was pleased to see that there is some <a href="http://caogadgets.com/media/mytaglist.com/apidoc.html">developer documentation</a> available for the web service API of the <a href="http://mytaglist.com/">mytaglist.com</a> backend server, along with the the source code for the management software I was complaining about earlier. The source code for the iOS and Android applications is also available if you email the company.<br />
<br />
<i><b>Update (12 Nov):</b> I emailed the manufacturer and they want to know why I want the source code, and for me to sign an NDA before they release it to me, which wasn't exactly what I was expecting from the <a href="http://caogadgets.com/media/mytaglist.com/apidoc.html">API documentation.</a> I'm not sure what to think about that...</i></div>
<div>
<br /></div>
<div>
The system does crucially rely on a back end server provided by the manufacturer, but the <a href="http://wirelesstags.myshopify.com/pages/known-issues-solutions">FAQ</a> tells us that if the tag manager can be flashed to point at a different server, and it's at least theoretically possible to run your own. The vendor then talks about licensing their own server side software, and promises that if their <a href="http://mytaglist.com/">mytaglist.com</a> server gets shut down that they'll release the code and allow you to update the tag manager to point at your own server. So I'm happy enough at that point.</div>
<div>
<br /></div>
<div>
<b>Summary</b></div>
<div>
<br /></div>
<div>
Just as I thought when I initially placed my order, someone has thought long and hard about this problem space, and it really shows in the quality and flexibility of the hardware if not the software. </div>
<div>
<br /></div>
<div>
I'm fairly sure the software issues are going to get ironed out eventually, at least for me the existence of some sort of documentation would probably solve most of the problems I'm having. Although I do think that for the average user the interface needs a good UX expert and a redesign. </div>
<div>
<br /></div>
<div>
<strike>I also hope to have my hands on the iOS app so I can use the tags as I originally intended, and enable push messaging to my phone, as that'll vastly increase the utility of the system for me.</strike></div>
<div>
<br /></div>
<div>
From poking around on the manufacturer's website it seems that there are probably more tag types coming, including current (for home energy monitoring?) and moisture (for detecting water leaks) sensor tags. I'll certainly go ahead and purchase those when they arrive.</div>
<div>
<br /></div>
<div>
Despite some of the criticism above, I am impressed with this system and would recommend it.<br />
<br />
<i style="font-weight: bold;">Update (23 Nov):</i> <i>My tags were suffering from spurious "out of range" and "reconnected" messages. So I have just sent all my tags back to CAO Gadgets for a <a href="https://wirelesstags.myshopify.com/blogs/news/6895552-wireless-tag-v1-3-update">firmware update</a>.</i></div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com17tag:blogger.com,1999:blog-7734900.post-2937169464950446942012-09-06T14:05:00.003+01:002012-09-11T10:34:55.016+01:00With conflicting stories, all we can believe is the data<br />
<div style="margin-bottom: 24px; text-align: center;">
<span style="color: #444444; font-family: Georgia, Bitstream Charter, serif;"><span style="line-height: 24px;"><i>This post was originally published on the <a href="http://radar.oreilly.com/2012/09/udid-data-analysis.html">O'Reilly Radar</a></i></span></span><br />
<span style="color: #444444; font-family: Georgia, Bitstream Charter, serif;"><span style="line-height: 24px;"><i>under the tile "Digging into the UDID Data"</i></span></span></div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<div style="line-height: 1.5; margin-bottom: 24px;">
Over the weekend the hacker group <a data-mce-href="http://en.wikipedia.org/wiki/Operation_AntiSec" href="http://en.wikipedia.org/wiki/Operation_AntiSec" style="color: #743399; line-height: 1.5;">Antisec</a> <a data-mce-href="http://www.wired.com/threatlevel/2012/09/hackers-release-1-million-apple-device-ids-allegedly-stolen-from-fbi-laptop/" href="http://www.wired.com/threatlevel/2012/09/hackers-release-1-million-apple-device-ids-allegedly-stolen-from-fbi-laptop/" style="color: #743399; line-height: 1.5;">released one million UDID records</a> that they claim to have obtained from an FBI laptop using a Java vulnerability. In reply <a data-mce-href="http://http://www.wired.com/threatlevel/2012/09/fbi-says-laptop-wasnt-hacked-never-possessed-file-of-apple-device-ids" href="http://http//www.wired.com/threatlevel/2012/09/fbi-says-laptop-wasnt-hacked-never-possessed-file-of-apple-device-ids" style="color: #743399; line-height: 1.5;">the FBI stated</a>:</div>
<blockquote style="font-style: italic; line-height: 1.5; padding: 0px 3em;">
<div style="line-height: 1.5; margin-bottom: 24px;">
The FBI is aware of published reports alleging that an FBI laptop was compromised and private data regarding Apple UDIDs was exposed. At this time there is no evidence indicating that an FBI laptop was compromised or that the FBI either sought or obtained this data.</div>
</blockquote>
<div style="line-height: 1.5; margin-bottom: 24px;">
Of course that statement leaves a lot of leeway. It could be the agent's personal laptop, and the data may well have been "property" of an <a data-mce-href="http://www.ncfta.net" href="http://www.ncfta.net/" style="color: #743399; line-height: 1.5;">another agency</a>. The wording doesn't even explicitly rule out the possibility that this was an agency laptop, they just say that right now they don't have any evidence to suggest that it was.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
This limited data release doesn't have much impact, but the possible release of the full dataset, which is <a data-mce-href="http://www.wired.com/threatlevel/2012/09/hackers-release-1-million-apple-device-ids-allegedly-stolen-from-fbi-laptop/" href="http://www.wired.com/threatlevel/2012/09/hackers-release-1-million-apple-device-ids-allegedly-stolen-from-fbi-laptop/" style="color: #743399; line-height: 1.5;">claimed to include</a> names, addresses, phone numbers and other identifying information, is far more worrying.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
While there are some almost <a data-mce-href="http://www.bbc.co.uk/news/technology-19491422" href="http://www.bbc.co.uk/news/technology-19491422" style="color: #743399; line-height: 1.5;">dismissing the issue out of hand</a>, the real issues here are: Where did the data originate? Which devices did it come from and what kind of users does this data represent? Is this data from a cross-section of the population, or a specifically targeted demographic? Does it originate within the law enforcement community, or from an external developer? What was the purpose of the data, and why was it collected?</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
With conflicting stories from all sides, the only thing we can believe is the data itself. The 40-character strings in the release <a data-mce-href="http://theiphonewiki.com/wiki/index.php?title=UDID" href="http://theiphonewiki.com/wiki/index.php?title=UDID" style="color: #743399; line-height: 1.5;">at least look like UDID</a> numbers, and anecdotally at least <a data-mce-href="https://twitter.com/peterkruse/status/242936275420717056" href="https://twitter.com/peterkruse/status/242936275420717056" style="color: #743399; line-height: 1.5;">we have a third-party confirmation</a> that this really is valid UDID data. We therefore have to proceed at this point as if this is real data. While there is a possibility that some, most, or all of the data is falsified, that's looking unlikely from where we're standing standing at the moment.</div>
</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
With that as the backdrop, the first action I took was to check the released data for my own devices and those of family members. Of the nine iPhones, iPads and iPod Touch devices kicking around my house, none of the UDIDs are in the leaked database. Of course there isn't anything to say that they aren't amongst the other 11 million UDIDs that haven't been released.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
With that done, I broke down the distribution of leaked UDID numbers by device type. Interestingly, considering the number of iPhones in circulation compared to the number of iPads, the bulk of the UDIDs were self-identified as originating on an iPad.</div>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-0xc_bif4zjY/UEe-T7-wCzI/AAAAAAAAH1c/SjgMAUQNUo8/s1600/Screen+Shot+2012-09-05+at+15.29.23.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="220" src="http://4.bp.blogspot.com/-0xc_bif4zjY/UEe-T7-wCzI/AAAAAAAAH1c/SjgMAUQNUo8/s400/Screen+Shot+2012-09-05+at+15.29.23.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Distribution of UDID by device type</i></td></tr>
</tbody></table>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
What does that mean? Here's one theory: If the leak originated from a developer <a data-mce-href="http://arstechnica.com/apple/2012/09/apple-denies-giving-ios-device-identifier-list-to-fbi/" href="http://arstechnica.com/apple/2012/09/apple-denies-giving-ios-device-identifier-list-to-fbi/" style="color: #743399; line-height: 1.5;">rather than directly from Apple</a>, and assuming that this subset of data is a good cross-section on the total population, and assuming that the leaked data originated with a single application ... then the app that harvested the data is likely a Universal application (one that runs on both the iPhone and the iPad) that is mostly used on the iPad rather than on the iPhone.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
The very low numbers of iPod Touch users might suggest either demographic information, or that the application is not widely used by younger users who are the target demographic for the iPod Touch, or alternatively perhaps that the application is most useful when a cellular data connection is present.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
The next thing to look at, as the only field with unconstrained text, was the Device Name data. That particular field contains a lot of first names, e.g. "Aaron's iPhone," so roughly speaking the distribution of first letters in the this field should give a decent clue as to the geographical region of origin of the leaked list of UDIDs. This distribution is of course going to be different depending on the predominant language in the region.</div>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-Tt99y0IuFQg/UEe-kzsDPSI/AAAAAAAAH1k/rQiiKKK4KN8/s1600/Screen+Shot+2012-09-05+at+16.41.08.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="205" src="http://2.bp.blogspot.com/-Tt99y0IuFQg/UEe-kzsDPSI/AAAAAAAAH1k/rQiiKKK4KN8/s400/Screen+Shot+2012-09-05+at+16.41.08.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Distribution of UDID by the first letter of the "Device Name" field</i></td></tr>
</tbody></table>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
The immediate stand out from this distribution is the predominance of device name strings starting with the letter "i." This can be ascribed to people who don't have their own name prepended to the Device Name string, and have named their device "iPhone," "iPad" or "iPod Touch."</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
The obvious next step was to compare this distribution with the relative frequency of first letters in words in the English language.</div>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-4b0blbNMxDY/UEfBCmsP_tI/AAAAAAAAH2E/LrT95bgOSSA/s1600/updated.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="218" src="http://3.bp.blogspot.com/-4b0blbNMxDY/UEfBCmsP_tI/AAAAAAAAH2E/LrT95bgOSSA/s400/updated.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Comparing the distribution of UDID by first letter of the "Device Name" field against the relative frequencies of the first letters of a word in the English language</i></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
The spike for the letter "i" dominated the data, so the next step was to do some rough and ready data cleaning.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
I dropped all the Device Name strings that started with the string "iP." That cleaned out all those devices named "iPhone," "iPad" and "iPod Touch." Doing that brought the number of device names starting with an "i" down from 159,925 to just 13,337. That's a bit more reasonable.</div>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-0ZknnvoEJp4/UEe_KBWJuII/AAAAAAAAH10/ZrXloIKg1E4/s1600/Screen+Shot+2012-09-05+at+17.27.25.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="216" src="http://2.bp.blogspot.com/-0ZknnvoEJp4/UEe_KBWJuII/AAAAAAAAH10/ZrXloIKg1E4/s400/Screen+Shot+2012-09-05+at+17.27.25.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Comparing the distribution of UDID by first letter of the "Device Name" field, ignoring all names that start with the string "iP", against the relative frequencies of the first letters of a word in the English language</i></td></tr>
</tbody></table>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
I had a slight over-abundance of "j," although that might not be statistically significant. However, the stand out was that there was a serious under-abundance of strings starting with the letter "t," which is interesting. Additionally, with my earlier data cleaning I also had a slight under-abundance of "i," which suggested I may have been too enthusiastic about cleaning the data.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
Looking at the <a data-mce-href="http://en.wikipedia.org/wiki/Letter_frequency#Relative_frequencies_of_letters_in_other_languages" href="http://en.wikipedia.org/wiki/Letter_frequency#Relative_frequencies_of_letters_in_other_languages" style="color: #743399; line-height: 1.5;">relative frequency of letters in languages other than English</a> it's notable that amongst them Spanish has a much lower frequency of the use of "t."</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
As the de facto second language of the United States, Spanish is the obvious next choice to investigate. If the devices are predominantly Spanish in origin then <a data-mce-href="https://twitter.com/markvillacampa/status/243381051639074816" href="https://twitter.com/markvillacampa/status/243381051639074816" style="color: #ff4b33; line-height: 1.5;">this could solve the problem</a> introduced by our data cleaning. In Spanish you would say "iPhone de Mark" rather than "Mark's iPhone."</div>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-j3Ux7_2hmCc/UEe_VopNN_I/AAAAAAAAH18/83M57rgPUsk/s1600/Screen+Shot+2012-09-05+at+20.34.55.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="217" src="http://3.bp.blogspot.com/-j3Ux7_2hmCc/UEe_VopNN_I/AAAAAAAAH18/83M57rgPUsk/s400/Screen+Shot+2012-09-05+at+20.34.55.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Comparing the distribution of UDID by first letter of the "Device Name" field, ignoring all names that start with the string "iP", against the relative frequencies of the first letters of a word in the Spanish language</i></td></tr>
</tbody></table>
<div style="color: #444444; font-family: Georgia, 'Bitstream Charter', serif; font-size: 16px; line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
However, that distribution didn't really fit either. While "t" was much better, I now had an under-abundance of words with an "e." Although it should be noted that, unlike our English language relative frequencies, the data I was using for Spanish is for letters in the entire word, rather than letters that begin the word. That's certainly going to introduce biases, perhaps fatal ones.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
Not that I can really make the assumption that there is only one language present in the data, or even that one language predominates, unless that language is English.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
At this stage it's obvious that the data is, at least more or less, of the right order of magnitude. The data probably shows devices coming from a Western country. However, we're a long way from the point where I'd come out and say something like " ... the device names were predominantly in English." That's not a conclusion I can make.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
I'd be interested in tracking down the <a data-mce-href="http://en.wikipedia.org/wiki/Arabic_Letter_Frequency" href="http://en.wikipedia.org/wiki/Arabic_Letter_Frequency" style="color: #743399; line-height: 1.5;">relative frequency of letters used in Arabic</a> when the language is transcribed into the Roman alphabet. While I haven't been able to find that data, I'm sure it exists somewhere. (Please drop a note in the comments if you have a lead.)</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
The next step for the analysis is to look at the names themselves. While I'm still in the process of mashing up something that will access U.S. census data and try and reverse geo-locate a name to a "most likely" geographical origin, <a data-mce-href="http://publicsector.experian.co.uk/Products/Mosaic%20Origins.aspx" href="http://publicsector.experian.co.uk/Products/Mosaic%20Origins.aspx" style="color: #743399; line-height: 1.5;">such services do already exist</a>. And I haven't really pushed the boundaries here, or even started a serious statistical analysis of the subset of data released by Antisec.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
This brings us to <a data-mce-href="http://petewarden.com" href="http://petewarden.com/" style="color: #743399; line-height: 1.5;">Pete Warden's</a> point that you <a data-mce-href="http://strata.oreilly.com/2011/05/anonymize-data-limits.html" href="http://strata.oreilly.com/2011/05/anonymize-data-limits.html" style="color: #743399; line-height: 1.5;">can't really anonymize your data</a>. The anonymization process for large datasets such as this is simply an illusion. As Pete <a data-mce-href="http://strata.oreilly.com/2011/05/anonymize-data-limits.html" href="http://strata.oreilly.com/2011/05/anonymize-data-limits.html" style="color: #743399; line-height: 1.5;">wrote</a>:</div>
<blockquote style="font-style: italic; line-height: 1.5; padding: 0px 3em;">
<div style="line-height: 1.5; margin-bottom: 24px;">
Precisely because there are now so many different public datasets to cross-reference, any set of records with a non-trivial amount of information on someone’s actions has a good chance of matching identifiable public records.</div>
</blockquote>
<div style="line-height: 1.5; margin-bottom: 24px;">
While this release in itself is fairly harmless, a number of "harmless" releases taken together — or cleverly cross-referenced with other public sources such as Twitter, Google+, Facebook and other social media — might well be more damaging. And that's ignoring the possibility that Antisec really might have names, addresses and telephone numbers to go side-by-side with these UDID records.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
The question has to be asked then, where did this data originate? While 12 million records might seem a lot, compared to the <a data-mce-href="http://en.wikipedia.org/wiki/IPhone#History_and_availability" href="http://en.wikipedia.org/wiki/IPhone#History_and_availability" style="color: #743399; line-height: 1.5;">number of devices sold</a> it's not actually that big a number. There are any number of iPhone applications with a 12-million-user installation base, and this sort of backend database could easily have been built up by an independent developer with a successful application who downloaded the device owner's contact details <a data-mce-href="http://www.cultofmac.com/173128/new-ios-6-privacy-settings-limit-access-to-photos-contact-calendars-and-more/" href="http://www.cultofmac.com/173128/new-ios-6-privacy-settings-limit-access-to-photos-contact-calendars-and-more/" style="color: #743399; line-height: 1.5;">before Apple started putting limitations</a> on that.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
Ignoring conspiracy theories, this dataset might be the result of a single developer. Although how it got into the FBI's possession and the why of that, if it was ever there in the first place, is another matter entirely.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
I'm going to go on hacking away at this data to see if there are any more interesting correlations, and I do wonder whether <a data-mce-href="http://en.wikipedia.org/wiki/Operation_AntiSec" href="http://en.wikipedia.org/wiki/Operation_AntiSec" style="color: #743399; line-height: 1.5;">Antisec</a> would consider a controlled release of the data to some trusted third party?</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
Much like the reaction to <a data-mce-href="http://radar.oreilly.com/2011/04/apple-location-tracking.html" href="http://radar.oreilly.com/2011/04/apple-location-tracking.html" style="color: #743399; line-height: 1.5;">#locationgate</a>, where some people were <a data-mce-href="http://crowdflow.net" href="http://crowdflow.net/" style="color: #743399; line-height: 1.5;">happy to volunteer their data</a>, if enough users are willing to self-identify, then perhaps we can get to the bottom of where this data originated and why it was collected in the first place.</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<em style="border: none; line-height: 1.5;">Thanks to <a data-mce-href="https://twitter.com/hmason" href="https://twitter.com/hmason" style="color: #743399; line-height: 1.5;">Hilary Mason</a>, <a data-mce-href="https://twitter.com/jsteeleeditor" href="https://twitter.com/jsteeleeditor" style="color: #743399; line-height: 1.5;">Julie Steele</a>, <a data-mce-href="https://twitter.com/ireneros" href="https://twitter.com/ireneros" style="color: #743399; line-height: 1.5;">Irene Ros</a>, <a data-mce-href="https://plus.google.com/112357111574249260299/about" href="https://plus.google.com/112357111574249260299/about" style="color: #743399; line-height: 1.5;">Gemma Hobson</a> and <a data-mce-href="https://twitter.com/MarkVillacampa" href="https://twitter.com/MarkVillacampa" style="color: #743399; line-height: 1.5;">Marcos Villacampa</a> for ideas, pointers to comparative data sources, and advice on visualisation of the data.</em><br />
<em style="border: none; line-height: 1.5;"><br /></em>
<em style="border: none; line-height: 1.5;"></em><br />
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; font-style: normal; line-height: 1.5;"><b style="font-style: italic;">Update: </b>In <a data-mce-href="https://plus.google.com/117841261693434574785/posts/Ni1AAFn27ZN" href="https://plus.google.com/117841261693434574785/posts/Ni1AAFn27ZN" style="color: #743399; line-height: 1.5;">response</a> to a post about this article on Google+, <a data-mce-href="https://plus.google.com/104522326787934519002/posts" href="https://plus.google.com/104522326787934519002/posts" style="color: #743399; line-height: 1.5;">Josh Hendrix</a> made the suggestion that I should look at word as well as letter frequency. It was a good idea, so I went ahead and wrote a quick script to do just that...</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">The top two words in the list are "iPad," which occurs 445,111 times, and "iPhone," which occurs 252,106 times. The next most frequent word is "iPod," but that occurs only 36,367 times. This result backs up my earlier result looking at distribution by device type.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Then there are various misspellings and mis-capitalisations of "iPhone," "iPad," and "iPod."</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">The first real word that isn't an Apple trademark is "Administrator," which occurs 10,910 times. Next are "David" (5,822), "John" (5,447), and "Michael" (5,034). This is followed by "Chris" (3,744), "Mike" (3,744), "Mark" (3,66) and "Paul" (3,096).</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Looking down the list of real names, as opposed to partial strings and tokens, the first female name doesn't occur until we're 30 places down the list — it's "Lisa" (1,732) with the next most popular female name being "Sarah" (1,499), in 38th place.</span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-hQZwVu5hg_k/UEuAydU-_VI/AAAAAAAAII8/lV7qpwDcmnY/s1600/4.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="148" src="http://1.bp.blogspot.com/-hQZwVu5hg_k/UEuAydU-_VI/AAAAAAAAII8/lV7qpwDcmnY/s400/4.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The top 100 names occurring in the UDID data</i></td></tr>
</tbody></table>
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><i><span style="border: none; line-height: 1.5;"></span></i></span></div>
<div style="display: inline !important; line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;">The word "Dad" occurs 1,074 times, with "Daddy" occurring 383 times. For comparison the word "Mum" occurs just 58 times, and "Mummy" just 33. "Mom" came in with 150 occurrences, and "mommy" with 30. The number of occurrences for "mum," "mummy," "mom," and "mommy" combined is 271, which is still very small compared to the combined total of 1,457 for "dad" and "daddy."</span></span></div>
</div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<br />
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;"><i>[<strong style="color: black; line-height: 1.5;">Updated:</strong> <a data-mce-href="https://twitter.com/gyardley/" href="https://twitter.com/gyardley/" style="color: #743399; line-height: 1.5;">Greg Yardly</a> <a data-mce-href="https://twitter.com/gyardley/status/243784351265984513" href="https://twitter.com/gyardley/status/243784351265984513" style="color: #743399; line-height: 1.5;">pointed out on Twitter</a> that I was being a bit British-centric in only looking for the words "mum" and "mummy," which is why I expanded the scope to include "mom" and "mommy."]</i></span></span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">There is a definite gender bias here, and I can think of at least a few explanations. The most likely is fairly simplistic: The application where the UDID numbers originated either appeals to, or is used more, by men.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Alternatively, women may be less likely to include their name in the name of their device, perhaps because amongst other things this name is used to advertise the device on wireless networks?</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Either way I think this definitively pins it down as a list of devices originating in an Anglo-centric geographic region.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Sometimes the simplest things work better. Instead of being fancy perhaps I should have done this in the first place. However this, combined with my previous results, suggest that we're looking at an English speaking, mostly male, demographic.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Correlating the top 20 or so names and with the list of most popular baby names (by year) all the way from the mid-'60s up until the mid-'90s (so looking at the most popular names for people between the ages of say 16 and 50) might give a further clue as to the exact demographic involved.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">Both <a data-mce-href="https://plus.google.com/112357111574249260299/posts" href="https://plus.google.com/112357111574249260299/posts" style="color: #743399; line-height: 1.5;">Gemma Hobson</a> and <a data-mce-href="http://radar.oreilly.com/julies" href="http://radar.oreilly.com/julies" style="color: #743399; line-height: 1.5;">Julie Steele</a> directed me toward the U.S. Social Security Administration's <a data-mce-href="http://www.ssa.gov/oact/babynames/decades/index.html" href="http://www.ssa.gov/oact/babynames/decades/index.html" style="color: #743399; line-height: 1.5;">Popular Baby Names By Decade</a> list. A quick and dirty analysis suggests that the UDID data is dominated by names that were most popular in the '70s and '80s. This maps well to my previous suggestion that the lack of iPod Touch usage might suggest that the demographic was older.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;">I'm going to do a <a data-mce-href="http://www.wolframalpha.com/input/?i=David" href="http://www.wolframalpha.com/input/?i=David" style="color: #743399; line-height: 1.5;">year-by-year breakdown</a> and some proper statistics later on, but we're looking at an application that's probably used by: English speaking males with an Anglo-American background in their 30s or 40s. It's most used on the iPad, and although it also works on the iPhone, it's used far less on that platform.</span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<div style="font-style: normal;">
<em style="border: none; line-height: 1.5;"><em style="border: none; line-height: 1.5;">Thanks to <a data-mce-href="https://plus.google.com/104522326787934519002/posts" href="https://plus.google.com/104522326787934519002/posts" style="color: #743399; line-height: 1.5;">Josh Hendrix</a>, and again to <a data-mce-href="https://plus.google.com/112357111574249260299/about" href="https://plus.google.com/112357111574249260299/about" style="color: #743399; line-height: 1.5;">Gemma Hobson</a> and <a data-mce-href="https://twitter.com/jsteeleeditor" href="https://twitter.com/jsteeleeditor" style="color: #743399; line-height: 1.5;">Julie Steele</a>, for ideas and pointers to sources for this part of the analysis.</em></em></div>
<div style="font-style: normal;">
<em style="border: none; line-height: 1.5;"><em style="border: none; line-height: 1.5;"><br /></em></em>
<em style="border: none; line-height: 1.5;"><em style="border: none; line-height: 1.5;"></em></em></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;"><b>Update: </b>A <a data-mce-href="http://intrepidusgroup.com/insight/2012/09/tracking-udid-src/" href="http://intrepidusgroup.com/insight/2012/09/tracking-udid-src/" style="color: #743399; line-height: 1.5;">really nice analysis</a> from David Schultz using the frequency of UDID duplicates and the names of those devices to track down the source of the leak. I really should of thought of that...</span></span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;">Interestingly however it does support my own analysis. <a data-mce-href="http://www.bluetoad.com/BlueToad/" href="http://www.bluetoad.com/BlueToad/" style="color: #743399; line-height: 1.5;">BlueToad</a> makes apps for magazine publishers, hence the predominance of of the iPad over the iPhone in my results, as those apps are more normally used on the iPad.</span></span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;">Also they seem to mostly market into the U.S., which supports my ethnicity findings, and looking at <a data-mce-href="http://www.coverstand.com" href="http://www.coverstand.com/" style="color: #743399; line-height: 1.5;">the list of titles</a> they curate, it does look like my demographics are more-or-less spot on as well. Those look like magazines marketed to men in their 30's and 40's to me...</span></span></div>
<div style="line-height: 1.5; margin-bottom: 24px;">
<span style="border: none; line-height: 1.5;"><span style="border: none; line-height: 1.5;">I'd actually been really confused about what type of app could possibly have that narrow a demographic, and this sort of clears up my confusion. Nice!</span></span></div>
</div>
</div>
</div>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com10tag:blogger.com,1999:blog-7734900.post-59877641591038022212012-08-30T12:49:00.000+01:002012-08-30T12:50:36.648+01:00Hardware Hacking for iOS Programmers<div style="text-align: center;">
<i>This post was originally published on <a href="http://www.josetteorama.com/hardware/hardware-hacking-for-ios-programmers/">Josetteorama</a>.</i></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
The arrival of the <a data-mce-href="http://www.apple.com/iphone/" href="http://www.apple.com/iphone/">iPhone</a> changed the whole direction of software development for mobile platforms, and has had a profound impact on the hardware design of the smart phones that have followed it.<br />
<br /></div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
Not only do these devices know where they are, they can tell you how they're being held, they are sufficiently powerful to overlay data layers on the camera view, and record and interpret audio data, and they can do all this in real time. These are not just smart phones, these are computers that just happen to be able to make phone calls.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-pZ-PxKLUQbc/UD9R7e_IyCI/AAAAAAAAHus/fZbExrHaoHU/s1600/screencap2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="222" src="http://1.bp.blogspot.com/-pZ-PxKLUQbc/UD9R7e_IyCI/AAAAAAAAHus/fZbExrHaoHU/s400/screencap2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Alasdair Allan demonstrating an Augmented Reality application</i></td></tr>
</tbody></table>
The arrival of the <a data-mce-href="http://developer.apple.com/library/ios/#documentation/ExternalAccessory/Reference/ExternalAccessoryFrameworkReference/_index.html" href="http://developer.apple.com/library/ios/#documentation/ExternalAccessory/Reference/ExternalAccessoryFrameworkReference/_index.html">External Accessory Framework</a> was seen, initially at least, as having the potential to open the iOS platform up to a host of external accessories and additional sensors. Sadly, little of the innovation people were expecting actually occurred, and while there are finally starting to be some interesting products arriving on the market, for the most part the External Accessory Framework is being used to support a fairly predictable range of audio and video accessories from big-name manufacturers.</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
<br /></div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
The reason for this lack of innovation is usually laid at the feet of Apple's <a data-mce-href="https://developer.apple.com/ipod/" href="https://developer.apple.com/ipod/">Made for iPod</a> (MFi) licensing program. To develop hardware accessories that connect to the iPod, iPhone, or iPad, you must be an MFi licensee.</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
Unfortunately, becoming a member of the MFi program is not as simple as signing up as an <a data-mce-href="https://developer.apple.com/programs/" href="https://developer.apple.com/programs/">Apple Developer</a>, and it is a fairly lengthy process. From personal experience I can confirm that the process of becoming an MFi licensee is not for the faint-hearted. And once you’re a member of the program, getting your hardware out of prototype stage and approved by Apple for distribution and sale is not necessarily a simple process.<br />
<br /></div>
<div style="text-align: center;">
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/Lz33JpLUdjQ?version=3&hl=en_US"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/Lz33JpLUdjQ?version=3&hl=en_US" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
</div>
<br />
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
However all that started to change with the <a data-mce-href="http://www.dailyack.com/2011/07/connect-your-iphone-to-real-world.html" href="http://www.dailyack.com/2011/07/connect-your-iphone-to-real-world.html">arrival of Redpark's serial cable</a>. As it's MFi approved for the hobbyist market it allows you to connect your iPhone to external hardware very simply, it also allows you to easily prototype new external accessories, bypassing a lot of the hurt you experience trying to do that wholly within the confines of the MFi program.</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
<br /></div>
<div style="text-align: center;">
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/xMo1RiFFA34?version=3&hl=en_US"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/xMo1RiFFA34?version=3&hl=en_US" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
</div>
<div style="text-align: center;">
<br /></div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
Another important part of that change was the <a data-mce-href="http://arduino.cc/" href="http://arduino.cc/">Arduino</a>. The Arduino, and the <a data-mce-href="http://radar.oreilly.com/2011/05/arduino-open-hardware-movement.html" href="http://radar.oreilly.com/2011/05/arduino-open-hardware-movement.html">open hardware movement that has grown up with it</a> and to a certain extent around it, is enabling a generation of high-tech tinkers to prototype new ideas with fairly minimal hardware knowledge.<br />
<br />
Every so often a piece of technology can become a lever that lets people move the world, just a little bit. The Arduino is one of those levers. While it started off as a project to give artists access to embedded microprocessors for interactive design projects, I think it’s going to end up in a museum as one of the building blocks of the modern world. It allows rapid, cheap prototyping for embedded systems. It turns what used to be fairly tough hardware problems into simpler software problems.<br />
<br />
Turning things into software problems makes things more scalable, it drastically reduces development time scales, and up front investment, and as the whole dot com revolution has shown, it leads to innovation. Every interesting hardware prototype to come along seems to boast that it is Arduino-compatible, or just plain built on top of an Arduino.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-AvwfLSOmTwI/UD9SEJ6aD9I/AAAAAAAAHvA/542JlBuKHD8/s1600/paduino.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://1.bp.blogspot.com/-AvwfLSOmTwI/UD9SEJ6aD9I/AAAAAAAAHvA/542JlBuKHD8/s400/paduino.jpeg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Controlling an Arduino directly from the iPad</i></td></tr>
</tbody></table>
I think the <a data-mce-href="http://radar.oreilly.com/2012/08/flying-cars-build-future-make-diy.html" href="http://radar.oreilly.com/2012/08/flying-cars-build-future-make-diy.html">next round of innovation</a> is going to take Silicon Valley, and the rest of us, back to its roots, and <a data-mce-href="http://radar.oreilly.com/2011/05/arduino-open-hardware-movement.html" href="http://radar.oreilly.com/2011/05/arduino-open-hardware-movement.html">that's hardware</a>. If you're a software person the things that are open and the things that are closed are changing. The skills needed to work with the technology are changing as well.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://3.bp.blogspot.com/-NMPyc7tpvg8/UD9R65jjvdI/AAAAAAAAHuo/ZqQtBJmmnw4/s1600/screencap1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="222" src="http://3.bp.blogspot.com/-NMPyc7tpvg8/UD9R65jjvdI/AAAAAAAAHuo/ZqQtBJmmnw4/s400/screencap1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;"><i>Alasdair demonstrating an Augmented Reality application</i></td></tr>
</tbody></table>
</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
At the start of October I'll be running <a data-mce-href="http://sensorworkshops.com/" href="http://sensorworkshops.com/">a workshop on iOS Sensors and External Hardware</a>. It's going to be hardware hacking for iOS programmers, and an opportunity for people to get their hands dirty both the internal sensors in the phone, and <a data-mce-href="http://radar.oreilly.com/2011/09/next-mobile-war-external-accessory.html" href="http://radar.oreilly.com/2011/09/next-mobile-war-external-accessory.html">with external hardware</a>.</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
<br /></div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
The <a data-mce-href="http://sensorworkshops.com/" href="http://sensorworkshops.com/" title="workshop">workshop</a> is intended to guide you through the start of that change, and get you hands-on with the hardware in your iPhone you've probably been ignoring until now. How to make use of the on-board sensors and combine them to build sophisticated location aware applications. But also how to extend the reach of these sensors by connecting your iOS device to external hardware.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-mipJZYPj0e4/UD9SJ-n4VXI/AAAAAAAAHvI/aneAcmZNOvE/s1600/iOS_blinking_the_BeagleBone_heartbeat_LED.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://4.bp.blogspot.com/-mipJZYPj0e4/UD9SJ-n4VXI/AAAAAAAAHvI/aneAcmZNOvE/s400/iOS_blinking_the_BeagleBone_heartbeat_LED.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Blinking the heartbeat LED of a BeagleBone from the iPhone</i></td></tr>
</tbody></table>
We'll look at three micro-controller platforms, <a data-mce-href="http://www.dailyack.com/2011/07/connect-your-iphone-to-real-world.html" href="http://www.dailyack.com/2011/07/connect-your-iphone-to-real-world.html">the Arduino</a> and <a data-mce-href="http://www.dailyack.com/2012/08/blinking-beaglebones-heartbeat-led-from.html" href="http://www.dailyack.com/2012/08/blinking-beaglebones-heartbeat-led-from.html">the BeagleBone and Raspberry Pi</a>, and get our hands dirty building simple applications to control the boards and gather measurements from sensors connected to it, directly from the iPhone. The <a data-mce-href="http://sensorworkshops.com/" href="http://sensorworkshops.com/">course</a> should give you the background to build your own applications independently, using the hottest location-aware technology yet for any mobile platform.</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
<br /></div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
<div style="text-align: justify;">
<em>The <a data-mce-href="http://sensorworkshops.com" href="http://sensorworkshops.com/">workshop</a> will be on <strong>Monday the 8th of October</strong> at the <a data-mce-href="http://sensorworkshops.com/pages/venue.html" href="http://sensorworkshops.com/pages/venue.html">Hoxton Hotel</a> in <strong>London </strong>at the heart of <a data-mce-href="http://en.wikipedia.org/wiki/East_London_Tech_City" href="http://en.wikipedia.org/wiki/East_London_Tech_City">Tech City</a>, and right next to <a data-mce-href="http://en.wikipedia.org/wiki/Old_Street_Roundabout" href="http://en.wikipedia.org/wiki/Old_Street_Roundabout">Silicon Roundabout</a>. I'm extending a discount to readers; 10% off the ticket price with <strong>discount code <a data-mce-href="http://iossensormasterclass.eventbrite.co.uk/?discount=OREILLY10" href="http://iossensormasterclass.eventbrite.co.uk/?discount=OREILLY10">OREILLY10</a></strong>. That makes the early bird ticket price just £449.10 (was £499), or if you miss the early bird deadline (the 1st of September) a full priced ticket still only £629.10 (£699).</em></div>
</div>
<div style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 13px; line-height: 19px; text-align: -webkit-auto;">
</div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
<br /></div>
<div class="separator" style="clear: both; font-family: Times; font-size: medium; line-height: normal; text-align: center;">
<a href="http://iossensormasterclass.eventbrite.co.uk/?discount=OREILLY10" imageanchor="1" onmouseout="document.register.src='http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg';" onmouseover="document.register.src='http://1.bp.blogspot.com/-A9WKWVbKeGo/UC0a7kFCxHI/AAAAAAAAHLY/NZezUcB8bpI/s1600/register_down.jpg';" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Register" border="0" height="82" name="register" src="http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg" title="Register for the Workshop" width="200" /></a></div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
Monday 8th October 2012</div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
<i>Hoxton Hotel, London</i></div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
Early Bird Price: £499 (until 1st Sept.)</div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
Normal Price: £699</div>
<div style="font-family: Times; font-size: medium; line-height: normal;">
<b>Save 10% with code OREILLY10</b></div>
<br />Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com18tag:blogger.com,1999:blog-7734900.post-54006792969320996642012-08-26T12:37:00.000+01:002012-08-26T12:44:17.236+01:00Now with added Beagle BoneAfter the last couple of days <a href="http://sensorworkshops.com/">my workshop in London on the 8th of October</a>, at the <a href="http://sensorworkshops.com/pages/venue.html">Hoxton Hotel</a>, now <a href="http://www.dailyack.com/2012/08/blinking-beaglebones-heartbeat-led-from.html">has added BeagleBone and Raspberry Pi</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='450' height='300' src='https://www.blogger.com/video.g?token=AD6v5dxOmqjsprGxyn1xbRswowGN5K3sciRjlurhXCQwwGbtHbkZR43XiLhlyD1bYWEbSF2GTvwGeftmqEM' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div style="text-align: center;">
<i><span style="font-size: x-small;">Blinking the BeagleBone's heartbeat LED using the iPhone</span></i><br />
<i><span style="font-size: x-small;"><br /></span></i></div>
We're going top go hands on in a small class setting, deep diving into the iOS internal sensors and how to connect your iPhone or iPad to external hardware. Everyone will get their hands dirty, and everyone will come away knowing more about both the iPhone hardware and how to work with external accessories. So come along and get your hands dirty playing with iPhone, Arduino and now the BeagleBone and Raspberry Pi and get <b>10% off the Early Bird ticket price</b> today only with <b>code BEAGLE10</b>.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://iossensormasterclass.eventbrite.co.uk/?discount=BEAGLE10" imageanchor="1" onmouseout="document.register.src='http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg';" onmouseover="document.register.src='http://1.bp.blogspot.com/-A9WKWVbKeGo/UC0a7kFCxHI/AAAAAAAAHLY/NZezUcB8bpI/s1600/register_down.jpg';" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Register" border="0" height="82" name="register" src="http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg" title="Register for the Workshop" width="200" /></a></div>
Monday 8th October 2012<br />
<i>Hoxton Hotel, London</i><br />
Early Bird Price: £499 (until 1st Sept.)<br />
Normal Price: £699<br />
<b>Save 10% with code BEAGLE10</b>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com8tag:blogger.com,1999:blog-7734900.post-32398493056596177242012-08-25T18:02:00.002+01:002012-08-26T15:20:11.963+01:00Blinking the BeagleBone's heartbeat LED from the iPhoneFollowing up on the work I was doing last night <a href="http://www.dailyack.com/2012/08/peertalk-and-beaglebone.html">connecting the iPhone to the BeagleBone</a> using <a href="http://rsms.me/peertalk/">PeerTalk</a>. I've now reached the blinking LED stage, which is more-or-less the <i>"Hello World"</i> stage of any bit of hardware hack.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='450' height='300' src='https://www.blogger.com/video.g?token=AD6v5dwOneA_UcM3oj0wf_-yVfoJJgivDlcIxsYjys5cF8LP2jtqvIX2WMH0I8HIm35IwKFflpkrcD-60KY' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div style="text-align: center;">
<i><span style="font-size: x-small;">Blinking the BeagleBone's heartbeat LED using the iPhone</span></i></div>
<div style="text-align: left;">
<br />
I've been having a great back-and-forth on Twitter with <a href="http://davidahouse.com/">David House</a> while hacking away with this project, who is working away as I type to get this working on the Raspberry Pi. It's been a lot of fun.<br />
<br /></div>
<div style="text-align: left;">
If you want to replicate this on the BeagleBone you should first download and build the <a href="http://rsms.me/peertalk/">PeerTalk</a> library, and then build and deploy the iOS and OSX example applications and <a href="http://rsms.me/peertalk/">get that up and running</a>.<br />
<br />
Then connect up and boot your BeagleBone. You'll need to power the board using a mains adapter as when you're compiling things it's possible you'll be drawing enough amperage that you're computer will turn off the USB port to protect itself, and as a result power down your BeagleBone. I had this happen to me a couple of times before I finally dug a mains adapter out of my office drawer. However since you're powering the board from the mains you'll also have to connect an Ethernet cable so that you can <i>ssh</i> <i>root@beaglebone.local</i> and log into the board over the network.<br />
<span style="font-family: inherit;"><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><br /></b></span>
<span style="font-family: inherit;"><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">1.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> Go ahead and login to your BeagleBone</span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> as root.</span></span><br />
<b style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;"><br /></b>
<b style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">2.</b><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;"> <a href="https://libusb.org/">Download</a>, build and install </span><i style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">libusb. V</i><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">ersion 1.0.9 builds, links and installs okay.</span><br />
<b style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;"><br /></b>
<b style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">3.</b><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;"> <a href="http://www.cmake.org/">Download</a>, build and install </span><i style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">cmake, </i><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">which you'll need to build <i>usbmuxd</i> later.</span><i style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;"> </i><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">You'll need to g</span><span style="background-color: white; font-family: inherit; line-height: 18px; text-align: -webkit-auto;">rab the latest Git nightly checkout as older release versions don't build, having problems with the stock <i>libbz2</i> compression on the BeagleBone.</span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">4.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> We also need </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">libplist, </i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">h</span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">owever this</span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> is available as part of the package management system on Ångström Linux, so all you need to do to install this is type </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">opkg install libplist-dev</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> at the prompt.</span></span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">5.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> <a href="http://cgit.sukimashita.com/usbmuxd.git/">Download</a>, build and install </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">usbmuxd</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">. Version 1.0.8 builds, links and installs okay, although you may beed to use </span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><i>ccmake</i> and configure by hand, rather than using <i>cmake,</i> as it can't seem to find the <i>libusb</i> include files that got installed into <i>/usr/local</i>.</span></span><br />
<span style="font-family: inherit;"><i><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /></i><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">6.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> Create a </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">usbmux</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> user</span><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> <i>groupadd -r usbmux -g 114</i></span><i><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> useradd -r -g usbmux -d / -s /sbin/nologin -c "usbmux user" -u 114 usbmux</span></i><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">7.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> As the BeagleBoard </span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">doesn't have syslog turned on by default, and you'll need it for debugging, turn on </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">syslogd</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> from the relevant script in </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">/etc/init.d</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">.</span></span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">8.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> Run up the </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">usbmux</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> deamon, by typing </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">usbmuxd -v -v </i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">at the prompt.</span></span><br />
<span style="font-family: inherit;"><i><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /></i><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">9.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> Plug your iPhone into the (host side) USB on your BeagleBoard</span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">, you should see some debug scrolling by in </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">/var/log/messages</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">.</span></span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">10.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> <a href="https://github.com/davidahouse/peertalk-python">Download</a> David House's </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">peertalk-python</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> and <a href="http://code.google.com/p/iphone-dataprotection/source/browse/usbmuxd-python-client/usbmux.py?r=3e6e6f047d7314e41dcc143ad52c67d3ee8c0859">its dependances</a>.</span></span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">11.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> On your iPhone start the <a href="http://rsms.me/peertalk/">PeerTalk client for iOS</a></span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">.</span></span><br />
<span style="font-family: inherit;"><br style="background-color: white; line-height: 18px; text-align: -webkit-auto;" /><b style="background-color: white; line-height: 18px; text-align: -webkit-auto;">12.</b><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> Start the python </span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;">client on the BeagleBone by typing</span><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> </span><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">python ./peertalk.py</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> at the prompt.</span></span><br />
<span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><span style="font-family: inherit;"><br /></span></span>
<span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><span style="font-family: inherit;">Type in a message at the prompt, and you should see something like this...</span></span><br />
<span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><span style="font-family: inherit;"><br /></span></span>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-eQPwmY38S34/UDkAJsFtCfI/AAAAAAAAHh8/wUhwZE7JSxI/s1600/IMG_0039.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://4.bp.blogspot.com/-eQPwmY38S34/UDkAJsFtCfI/AAAAAAAAHh8/wUhwZE7JSxI/s400/IMG_0039.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Bi-directional communication between the iPhone and the BeagleBone via USB</i></td></tr>
</tbody></table>
From there it's pretty trivial to replicate my <i>"Hello World"</i> example, just by hacking around with David's code and toggling the heartbeat LED when the BeagleBone receives any messages.<br />
<br />
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">def</span> run(<span style="color: #9c2663;">self</span>):</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> framestructure = struct.Struct(<span style="color: #8a1b1b;">"! I I I I"</span>)</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> ledOn</span><span style="font-size: xx-small;"> =</span><span style="color: #8a1b1b; font-size: xx-small;">'echo 1 > /sys/class/leds/beaglebone::usr0/brightness'</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> ledOff</span><span style="font-size: xx-small;"> =</span><span style="color: #8a1b1b; font-size: xx-small;">'echo 0 > /sys/class/leds/beaglebone::usr0/brightness'</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> </span><span style="font-size: xx-small;">i = </span><span style="color: #0433ff; font-size: xx-small;">0</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">while</span> <span style="color: #9c2663;">self</span>._running:</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">try</span>:</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> msg = <span style="color: #9c2663;">self</span>._psock.recv(<span style="color: #0433ff;">16</span>)</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">if</span> <span style="color: #571d9a;">len</span>(msg) > <span style="color: #0433ff;">0</span>:</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> frame = framestructure.unpack(msg)</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> size = frame[<span style="color: #0433ff;">3</span>]</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> msgdata = <span style="color: #9c2663;">self</span>._psock.recv(size)</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">print</span> <span style="color: #8a1b1b;">"Received: %s"</span> % msgdata</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">if</span> i == <span style="color: #0433ff;">0</span>:</span></div>
<div style="color: #8a1b1b; font-family: Monaco;">
<span style="font-size: xx-small;"><span style="color: black;"> os.system(ledOn</span><span style="color: black;">)</span></span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> i = <span style="color: #0433ff;">1</span></span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">else</span>:</span></div>
<div style="color: #8a1b1b; font-family: Monaco;">
<span style="font-size: xx-small;"><span style="color: black;"> os.system(</span></span><span style="color: black; font-size: xx-small;">ledOff</span><span style="font-size: xx-small;"><span style="color: black;">)</span></span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> i = <span style="color: #0433ff;">0</span></span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">except</span>:</span></div>
<div style="font-family: Monaco;">
<span style="font-size: xx-small;"> <span style="color: #9c2663;">pass</span></span></div>
<br />
Which gets you to this point...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://3.bp.blogspot.com/-SfmOLCdnF8Y/UDkBY1i_8XI/AAAAAAAAHiE/xRTuVdkgkvk/s1600/IMG_0044.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://3.bp.blogspot.com/-SfmOLCdnF8Y/UDkBY1i_8XI/AAAAAAAAHiE/xRTuVdkgkvk/s400/IMG_0044.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;"><i>Toggling the BeagleBone heartbeat LED with my iPhone over USB.</i></td></tr>
</tbody></table>
Which is pretty much where I've reached right now. Next steps is a proper application on the iOS end of things with more generic control of the BeagleBone's header pins, and a more flexible Python backend on the BeagleBone itself...<br />
<div>
<br />
<b>Update: </b><a href="http://davidahouse.com/">David House</a> has managed to get everything up and working on the Raspberry Pi. The only changes from the above is that you should grab <i>libplist</i> using <i>apt-get</i> rather than <i>opkg</i>, and since you won't be logged in as root you should remember to <i>sudo </i><i style="background-color: white; line-height: 18px; text-align: -webkit-auto;">usbmuxd -v -v</i><span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"> when you start the USB daemon. Apart from that, you should be good to go...</span><br />
<span style="background-color: white; line-height: 18px; text-align: -webkit-auto;"><br /></span>
<img src="http://a0.twimg.com/profile_images/1720705802/IMG_0626_-_Version_2_normal.jpg" style="background-color: white; float: left; font-family: Helvetica; height: 48px; margin: 8px 8px 3px; text-align: -webkit-auto; width: 48px;" /><span style="font-family: inherit;"><b style="background-color: white; text-align: -webkit-auto;">David House (<a href="https://twitter.com/davidahouse">@davidahouse</a>)</b><br style="background-color: white; text-align: -webkit-auto;" /><a href="https://twitter.com/davidahouse/status/239442760229330945" style="background-color: white; text-align: -webkit-auto;">25/08/2012 20:22</a><br style="background-color: white; text-align: -webkit-auto;" /><span style="background-color: white; text-align: -webkit-auto;"><i>Video of iPhone controlling LED on Raspberry Pi.</i></span></span><br />
<span style="font-family: inherit;"><span style="background-color: white; text-align: -webkit-auto;"><br /></span></span></div>
</div>
<center>
<table><tbody>
<tr><td align="center"><object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/M7xaOeUutKY?version=3&hl=en_US"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/M7xaOeUutKY?version=3&hl=en_US" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object></td></tr>
<tr><td align="center"><span style="font-family: inherit; font-size: x-small;"><i>Controlling a LED connected to a GPIO pin on the Raspberry Pi with an iPhone</i></span></td></tr>
</tbody></table>
<br />
</center>
<center style="text-align: left;">
<b>Update:</b> Come along to my workshop in <b>London</b> on the <b>8th of October</b> and get your hands dirty playing with iPhone, Arduino and now the BeagleBone and Raspberry Pi. Get <b>10% off the Early Bird ticket price</b> today only with <b>code BEAGLE10</b>.</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
<div style="text-align: -webkit-auto;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://iossensormasterclass.eventbrite.co.uk/?discount=BEAGLE10" imageanchor="1" onmouseout="document.register.src='http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg';" onmouseover="document.register.src='http://1.bp.blogspot.com/-A9WKWVbKeGo/UC0a7kFCxHI/AAAAAAAAHLY/NZezUcB8bpI/s1600/register_down.jpg';" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Register" border="0" height="82" name="register" src="http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg" title="Register for the Workshop" width="200" /></a></div>
<div style="text-align: -webkit-auto;">
Monday 8th October 2012</div>
<div style="text-align: -webkit-auto;">
<i>Hoxton Hotel, London</i></div>
<div style="text-align: -webkit-auto;">
Early Bird Price: £499 (until 1st Sept.)</div>
<div style="text-align: -webkit-auto;">
Normal Price: £699</div>
<div style="text-align: -webkit-auto;">
<b>Save 10% with code BEAGLE10</b><br />
<b><br /></b>
<b style="text-align: left;">Update: </b><a href="http://davidahouse.com/" style="text-align: left;">David House</a> has<span style="text-align: left;"> just updated his <a href="https://github.com/davidahouse/PiTalk">Github repository</a> with a better description of what he did to get the iPhone to control the Raspberry Pi's GPIO pins.</span><br />
<table style="background-color: white; font-family: Helvetica;"><tbody>
<tr><td><img src="http://a0.twimg.com/profile_images/1720705802/IMG_0626_-_Version_2_normal.jpg" style="float: left; height: 48px; margin: 8px 8px 3px; width: 48px;" /><span style="font-family: inherit;"><b><span style="font-family: inherit;">David House (<a href="https://twitter.com/davidahouse">@davidahouse</a>)</span></b><br /><a href="https://twitter.com/davidahouse/status/239703979217547265">26/08/2012 13:40</a><br /><i><a href="https://twitter.com/@aallan">@aallan</a> I just updated my <a href="https://github.com/davidahouse/PiTalk">github repo</a> with a better description with attributions. Had a blast working with you...</i></span><br />
<span style="font-family: inherit;"><i><br /></i></span>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-ROj_QFEjUC0/UDovB5697dI/AAAAAAAAHos/KCJw9A-2BYk/s1600/iPhoneToPi2.JPG" imageanchor="1" style="font-family: Times; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="298" src="http://2.bp.blogspot.com/-ROj_QFEjUC0/UDovB5697dI/AAAAAAAAHos/KCJw9A-2BYk/s400/iPhoneToPi2.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i style="font-size: small; text-align: -webkit-auto;">Controlling a LED connected to a GPIO pin on the Raspberry Pi with an iPhone</i></td></tr>
</tbody></table>
<span style="font-size: x-small;"><i><br /></i></span></td></tr>
</tbody></table>
</div>
</center>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com41tag:blogger.com,1999:blog-7734900.post-59486479117313283712012-08-25T03:45:00.000+01:002012-08-29T21:20:30.195+01:00PeerTalk and the BeagleBoneEarlier today I came across an excellent bit of wizardry by <a href="http://rsms.me/about">Rasmus Andersson</a> called <a href="http://rsms.me/peertalk/">PeerTalk</a>. It's a Objective-C library allowing you to communicate between your iPhone and your Mac over the USB dock cable using TCP sockets.<br />
<br />
<center>
<table><tbody>
<tr><td align="center"><object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/kQPWy8N0mBg?version=3&hl=en_US"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/kQPWy8N0mBg?version=3&hl=en_US" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object>
</td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>PeerTalk Demo</i><br />
<i><br /></i></td></tr>
</tbody></table>
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
My immediate thought was that if this really only depended on having USB host mode capability at the far end, the same mechanism should be able to be used to talk to something like the <a href="http://beagleboard.org/bone/">BeagleBone</a>, or the <a href="http://www.raspberrypi.org/">Raspberry Pi</a>, not just your Mac. This would allow you connect your phone directly to the micro controller board and to drive hardware directly, a lot like the <a href="http://www.dailyack.com/2011/07/connect-your-iphone-to-real-world.html">Redpark cable</a> but bypassing Apple's <a href="http://developer.apple.com/library/ios/#documentation/ExternalAccessory/Reference/ExternalAccessoryFrameworkReference/_index.html">External Accessory</a> framework. </center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-OANEdaDN34s/UDg17WhK5vI/AAAAAAAAHfo/Vo6XmaxevvI/s1600/IMG_0035.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-OANEdaDN34s/UDg17WhK5vI/AAAAAAAAHfo/Vo6XmaxevvI/s400/IMG_0035.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Yup, this is going to be useful...</i></td></tr>
</tbody></table>
<center style="text-align: left;">
So I started digging around inside the source code to see if it depended on anything that was going to be specific to OS X, it became apparent that <a href="http://rsms.me/peertalk/" style="text-align: -webkit-auto;">PeerTalk</a> was mostly some really nice socket code sitting on top of the USB Multiplex Daemon (<i>usbmuxd</i>). This bit of software is in charge of talking to your iPhone over USB and coordinating access to its services by other applications. Effectively this is what iTunes and Xcode use to talk to your phone when you plug it into your Mac's USB port.</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
So any device that wants to talk to the iPhone using this method needs <i>usbmuxd</i>. Fortunately for me there are a number of people that have worked oout <a href="http://www.libimobiledevice.org/">how to talk to the iPhone from Linux</a>, and there is a working <i><a href="http://cgit.sukimashita.com/usbmuxd.git/">usbmuxd</a></i> for Linux.</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-EWFzTg0poD8/UDg4qxFUJOI/AAAAAAAAHgA/hDl9SdqhsjU/s1600/IMG_6064.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://1.bp.blogspot.com/-EWFzTg0poD8/UDg4qxFUJOI/AAAAAAAAHgA/hDl9SdqhsjU/s400/IMG_6064.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The BeagleBone</i></td></tr>
</tbody></table>
<center style="text-align: left;">
As well as a few other dependences which aren't present on the stock <a href="http://www.angstrom-distribution.org/">Ångström Linux</a> distribution on my BeagleBone, or even packages via <i>opkg</i>, building <i>usbmuxd</i> on my BeagleBone requires <i><a href="https://libusb.org/wiki/libusb-1.0">libusb</a></i> and <a href="http://www.cmake.org/"><i>cmake</i></a>. So before building <i>usbmuxd</i>, I had to build <i>cmake</i>, which meant resolving some problems with the stock compression libraries that shipped with Ångström.</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
However several hours later. after enough waiting around for software to build to convince me that before doing any serious development on the BeagleBone I really had to build an ARMv7 toolchain on my Mac to cross-compile things instead of building them directly on the board....</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://2.bp.blogspot.com/-D4RzArLKP14/UDg2_kpek9I/AAAAAAAAHfw/k_gpfTHE4OI/s1600/IMG_0036.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-D4RzArLKP14/UDg2_kpek9I/AAAAAAAAHfw/k_gpfTHE4OI/s400/IMG_0036.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;"><i> The iPhone talking directly to my BeagleBone using PeerTalk</i></td></tr>
</tbody></table>
</center>
<center style="text-align: left;">
...I managed to get a simple <i>"hello"</i> from my iPhone to the BeagleBone and then via <i>screen</i> to my Mac using port forwarding and that old stand by, <i>telnet</i>.</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
While I was hacking away on getting this working, I wasn't alone. <a href="http://davidahouse.com/">David House</a> was looking down some of the same back alleyways to get PeerTalk talking to his Raspberry Pi, and we batted the problem back and forth on Twitter while waiting for code to compile well into the night...</center>
<center style="text-align: left;">
<br /></center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
The next step is to put together a client on the BeagleBone sitting on top of <i>usbmuxd</i> that'll talk natively to the PeerTalk on iOS. Since I've got the source code of both ends, this isn't going to be too hard. I'll probably put something together in Python.</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
</center>
<center style="text-align: left;">
More soon...</center>
<center style="text-align: left;">
<b><br /></b></center>
<center style="text-align: left;">
<b>Update:</b> Following on from this I pushed forward till I managed to <a href="http://www.dailyack.com/2012/08/blinking-beaglebones-heartbeat-led-from.html">blink the BeagleBone's heartbeat LED from the iPhone</a> which is, more-or-less, the "<i>Hello World"</i> stage of any hardware hack...</center>
Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com9tag:blogger.com,1999:blog-7734900.post-23463135524963857272012-08-19T14:31:00.001+01:002012-08-19T14:32:37.812+01:00A drawer full of phones...<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;">I had a huge clear out of my home office this weekend, including the draw full of old mobile phones. A free copy of the second edition of <a href="http://shop.oreilly.com/product/0636920018490.do">Learning iOS Programming</a></span><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"> to the first person that can successfully identify them all...</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-EoFLtsTRQLU/UDDqKYawkmI/AAAAAAAAHOg/vyryIBSnA5o/s1600/IMG_6556.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://1.bp.blogspot.com/-EoFLtsTRQLU/UDDqKYawkmI/AAAAAAAAHOg/vyryIBSnA5o/s400/IMG_6556.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>A collection of mobile phones, but the iPhone changed everything..?</i></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
Answers accepted on the relevant <a href="https://plus.google.com/117841261693434574785/posts/LJFW42Kx9gb">thread on Google+</a> only....</div>
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px; line-height: 18px;"><br /></span>Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com10tag:blogger.com,1999:blog-7734900.post-80915238295665547162012-08-16T17:10:00.000+01:002012-08-16T17:32:37.465+01:00iOS Sensors and External Hardware MasterclassI'm going to be down in London for <a href="http://oreilly.com/">O'Reilly</a>'s <a href="http://strataconf.com/">Strata</a> conference in October, key-noting on the Tuesday morning, talking about the <a href="http://radar.oreilly.com/2012/03/hidden-data-exhaust-leakage-location.html">hidden data that follows us around</a> and how I've leveraged that for my own advantage. I'll also be talking, along with my colleague <a href="https://emps.exeter.ac.uk/staff/zmw201/">Zena Wood</a> from Exeter, about <a href="http://strataconf.com/strataeu/public/schedule/detail/25816">People Watching with Machine Learning</a> and using modern smart phones, like the iPhone, to do interesting sociology. It should be good.<br />
<div>
<br /></div>
<div>
<div>
<div>
However I'll be kicking around town for most of the following week afterwards, talking to various people. But I had a gap, a big gap, at the beginning of that week. So I've decided to try and interesting experiment...</div>
</div>
</div>
<div>
<br /></div>
<div>
I've often argued that both the increasingly rich sensor suite and the ability to <a href="http://radar.oreilly.com/2011/09/next-mobile-war-external-accessory.html">easily connect today's smart phones to external hardware</a>, and sensors, make them an amazing lever on the world. It's something I've focused on a lot over the last year or so.<br />
<div style="text-align: center;">
<br /></div>
</div>
<div style="text-align: center;">
<object height="253" width="450"><param name="movie" value="http://www.youtube.com/v/xMo1RiFFA34?version=3&hl=en_US"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="http://www.youtube.com/v/xMo1RiFFA34?version=3&hl=en_US" type="application/x-shockwave-flash" width="450" height="253" allowscriptaccess="always" allowfullscreen="true"></embed></object></div>
<div style="text-align: center;">
<i>A conversation with Dale Dougherty and Alasdair Allan</i></div>
<br />
While I've run a lot of conferences and workshops over the years, it's always been on someone else's dime. Time to put my money where my mouth is, I'm going to <a href="http://sensorworkshops.com/">run a workshop</a>.<br />
<br />
In fact I'm going to run a master-class on <i><a href="http://sensorworkshops.com/">iOS Sensors and External Hardware</a></i>. This is going to be hardware hacking for iOS programmers. It's going to be hands on, bring your Mac, bring your iPhone and make sure you've got Xcode set up so that you can deploy apps onto your device. It'll be a small group, no more than twenty, and I'll be doing a bunch of live coding.<br />
<br />
We'll start the day talking about the internal sensors: the accelerometer, magnetometer, gyroscope and how to combine these into sophisticated applications. Here you'll really get the benefit of my <a href="http://www.astro.ex.ac.uk/people/aa/">physics background</a>, because I can take you under the user friendly skin Apple have put on top of these sensors as part of the iOS SDK and hopefully give you a decent idea of what their limitations are and how they work.<br />
<br />
Then we'll move on to talk about how to extend the reach of the on-board sensors by connecting your iPhone to external hardware. We'll look at how to connect the <a href="http://arduino.cc/">Arduino</a> micro-controller platform to your iOS device, and build simple applications to control the board and gather measurements from sensors connected to it, directly from iOS. This course will give you the background to build your own applications independently, using the hottest location-aware technology yet for any mobile platform.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-nCBC0ZUqwqY/UC0Z1EJdc6I/AAAAAAAAHLI/DVxgJ84ysEY/s1600/paduino.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://4.bp.blogspot.com/-nCBC0ZUqwqY/UC0Z1EJdc6I/AAAAAAAAHLI/DVxgJ84ysEY/s400/paduino.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>iPad controlling an Arduino board via the Redpark cable</i></td></tr>
</tbody></table>
You'll take away with you an <a href="http://www.makershed.com/New_Arduino_Uno_Revision_3_p/mksp11.htm">Arduino Uno</a> board, a Redpark <a href="http://www.makershed.com/Redpark_TTL_Cable_for_iOS_p/msrp03.htm">TTL Serial Cable for iOS</a>, and everything you need to connect your iPhone to your new micro-controller. You'll also receive a copy of my books <a href="http://shop.oreilly.com/product/0636920021162.do">Basic Sensors in iOS</a> and <a href="http://shop.oreilly.com/product/0636920021179.do">iOS Sensor Apps with Arduino</a>.<br />
<br />
The <a href="http://sensorworkshops.com/">workshop</a> will be on <b>Monday the 8th of October</b> <a href="http://sensorworkshops.com/pages/venue.html">at the Hoxton Hotel</a> right next to London's Silicon Roundabout. I've <a href="http://sensorworkshops.com/">opened registration</a> and I'm offering <b>30% off</b> the ticket price until the 1st of September. Sign up early, and sign up often.<br />
<br />
More about exactly what I'm going to be talking about on the <a href="http://sensorworkshops.com/">workshop's own website</a>. I've done similar things on smaller scales in the past, but this should be a lot of fun. Hope to see at least some of you there...<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://sensorworkshops.com/" imageanchor="1" onmouseout="document.register.src='http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg';" onmouseover="document.register.src='http://1.bp.blogspot.com/-A9WKWVbKeGo/UC0a7kFCxHI/AAAAAAAAHLY/NZezUcB8bpI/s1600/register_down.jpg';" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Register" border="0" height="82" name="register" src="http://2.bp.blogspot.com/-vAYa3kvCTvQ/UC0aIQwsSqI/AAAAAAAAHLQ/Zfw86_G5d1E/s200/register.jpg" title="Register for the Workshop" width="200" /></a></div>
Monday 8th October 2012<br />
<i>Hoxton Hotel, London</i><br />
Early Bird Price: £499 (until 1st Sept.)<br />
Normal Price: £699<br />
<br />Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com7tag:blogger.com,1999:blog-7734900.post-83705888439225229192012-08-15T17:56:00.003+01:002012-08-15T18:00:18.404+01:00Mining the astronomical literature<div style="text-align: center;">
<i>This post was originally published on the <a href="http://radar.oreilly.com/2012/08/data-mining-the-literature.html">O'Reilly Radar</a>.</i></div>
<br />
There is a <a href="http://www.guardian.co.uk/science/2012/jun/19/open-access-academic-publishing-finch-report?intcmp=239">huge debate</a> right now about making academic literature freely accessible and moving toward open access. But what would be possible if people stopped talking about it and just dug in and got on with it?<br />
<br />
<a href="http://nasa.gov/" title="NASA">NASA's</a> <a href="http://adswww.harvard.edu/" title="Astrophysics Data System">Astrophysics Data System</a> (ADS), hosted by the <a href="http://www.cfa.harvard.edu/sao/" title="Smithsonian Astrophysical Observatory">Smithsonian Astrophysical Observatory</a> (SAO), has quietly been working away since the mid-'90s. Without much, if any, fanfare amongst the other disciplines, it has moved astronomers into a world where access to the literature is just a given. It's something they don't have to think about all that much.<br />
<br />
The <a href="http://adsabs.harvard.edu/abstract_service.html">ADS service</a> provides access to abstracts for virtually all of the astronomical literature. But it also provides access to the full text of more than half a million papers, going right back to the start of peer-reviewed journals in the 1800s. The service has links to online data archives, along with reference and citation information for each of the papers, and it's all <a href="http://adsabs.harvard.edu/abstract_service.html" title="searchable">searchable</a> and downloadable.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<a href="https://dl.dropbox.com/u/2068615/Orbiting%20Frog/publishing-rates.html"><img alt="Number of papers published in the three main astronomy journals each year" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-1-number-of-papers.png" style="margin-bottom: 15px;" width="450" /></a>
<em>Number of papers published in the three main astronomy journals each year. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a></em></div>
</div>
The existence of the <a href="http://adsabs.harvard.edu/abstract_service.html">ADS</a>, along with the <a href="http://arxiv.org/">arXiv</a> pre-print server, has meant that most astronomers haven't seen the inside of a brick-built library since the late 1990s.<br />
<br />
It also makes astronomy almost uniquely well placed for interesting data mining experiments, experiments that hint at what the rest of academia could do if they followed astronomy's lead. The fact that the discipline's literature has been scanned, archived, indexed and catalogued, and placed behind a RESTful API makes it a treasure trove, both for hypothesis generation and sociological research.<br />
<br />
For example, the <a href="http://dotastronomy.com/" title=".Astronomy">.Astronomy</a> series of conferences is a small workshop that brings together the best and the brightest of the technical community: researchers, developers, educators and communicators. Billed as <em>"20% time for astronomers,"</em> it gives these people space to think about how the new technologies affect both how research and communicating research to their peers and to the public is done.<br />
<br />
It should perhaps come as little surprise that one of the more interesting projects to come out of a hack day held as part of this year's .Astronomy meeting <a href="http://www.haus-der-astronomie.de/en/">in Heidelberg</a> was work by <a href="http://orbitingfrog.com/">Robert Simpson</a>, <a href="http://icg.port.ac.uk/~mastersk/">Karen Masters</a> and <a href="http://sarahaskew.net/">Sarah Kendrew</a> that focused on data mining the astronomical literature.<br />
<br />
The team <a href="http://orbitingfrog.com/post/27983055767/mining-the-astronomical-literature">grabbed and processed</a> the titles and abstracts of all the papers from the <a href="http://iopscience.iop.org/0004-637X/">Astrophysical Journal</a> (ApJ), <a href="http://www.aanda.org/">Astronomy & Astrophysics</a> (A&A), and the <a href="http://eu.wiley.com/WileyCDA/WileyTitle/productCd-MNR.html">Monthly Notices of the Royal Astronomical Society</a> (MNRAS) since each of those journals started publication — and that's 1827 in the case of MNRAS.<br />
<br />
By the end of the day, they'd found some interesting results showing how various terms have trended over time. The results were similar to what's found in Google Books' <a href="http://books.google.com/ngrams/graph?content=Astronomy%2C+Astrophysics&year_start=1800&year_end=2000&corpus=0&smoothing=3">Ngram Viewer</a>.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<a href="https://dl.dropbox.com/u/2068615/Orbiting%20Frog/sample-trends.html"><img alt="The relative popularity of the names of telescopes in the literature" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-3-telescopes.png" style="margin-bottom: 15px;" width="450" /></a>
<em>The relative popularity of the names of telescopes in the literature. Hubble, Chandra and Spitzer seem to have taken turns in hogging the limelight, much as COBE, WMAP and Planck have each contributed to our knowledge of the cosmic microwave background in successive decades. References to Planck are still on the rise. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a>.</em></div>
</div>
After the meeting, however, Robert has taken his <a href="http://orbitingfrog.com/post/27983055767/mining-the-astronomical-literature">initial results</a> and explored the astronomical literature and his new corpus of data on the literature. He's explored various visualisations of the data, including <a href="http://orbitingfrog.com/post/28143196783/more-astronomy-data-mining-its-word-matrix-time">word matrixes</a> for related terms and for various <a href="http://orbitingfrog.com/post/28434621487/astrochemistry-word-matrix">astro-chemistry</a>.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<a href="http://dl.dropbox.com/u/2068615/Orbiting%20Frog/MAtrix/matrix.html"><img alt="Correlation between terms related to Active Galactic Nuclei" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-4-agn.png" style="margin-bottom: 15px;" width="450" /></a>
<em>Correlation between terms related to Active Galactic Nuclei (AGN). The opacity of each square represents the strength of the correlation between the terms. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a>.</em></div>
</div>
He's also taken a look at <a href="http://orbitingfrog.com/post/28714839175/authorship-in-astronomy">authorship in astronomy</a> and is starting to find some interesting trends.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<img alt="Fraction of astronomical papers published with one, two, three, four or more authors" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-5-authors.png" style="margin-bottom: 15px;" width="450" />
<em>Fraction of astronomical papers published with one, two, three, four or more authors. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a></em></div>
</div>
You can <a href="http://orbitingfrog.com/post/28714839175/authorship-in-astronomy">see</a> that single-author papers dominated for most of the 20th century. Around 1960, we see the decline begin, as two- and three-author papers begin to become a significant chunk of the whole. In 1978, author papers become more prevalent than single-author papers.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<img alt="Compare the number of active research astronomers to the number of papers published each year" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-6-active-researchers.png" style="margin-bottom: 15px;" width="450" />
<em>Compare the number of "active" research astronomers to the number of papers published each year (across all the major journals). CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a>.</em></div>
</div>
Here we see that people begin to outpace papers in the 1960s. This may reflect the fact that as we get more technical as a field, and more specialised, it takes more people to write the same number of papers, which is a sort of interesting result all by itself.
<br />
<h2>
<br /></h2>
<h2>
Behind the project and what lies ahead</h2>
<br />
I recently talked with <a href="http://orbitingfrog.com/">Rob</a> about the work he, <a href="http://icg.port.ac.uk/~mastersk/">Karen Masters</a>, and <a href="http://sarahaskew.net/">Sarah Kendrew</a> did at the meeting, and the work he's been doing since with the newly gathered data.<br />
<br />
<strong>What made you think about data mining the <a href="http://adsabs.harvard.edu/abstract_service.html">ADS</a>?</strong><br />
<br />
<strong>Robert Simpson:</strong> At the <a href="http://dotastronomy.com/">.Astronomy</a> 4 Hack Day in July, <a href="http://sarahaskew.net/">Sarah Kendrew</a> had the idea to try to do an astronomy version of <a href="http://www.brainscanr.com/">BrainSCANr</a>, a project that generates new hypotheses in the neuroscience literature. I've had a go at mining <a href="http://adsabs.harvard.edu/abstract_service.html">ADS</a> and <a href="http://arxiv.org/">arXiv</a> before, so it seemed like a great excuse to dive back in.<br />
<br />
<strong>Do you think there might be actual science that could be done here?</strong><br />
<br />
<strong>Robert Simpson:</strong> Yes, in the form of finding questions that were unexpected. With such large volumes of peer-reviewed papers being produced daily in astronomy, there is a lot being said. Most researchers can only try to keep up with it all — my daily RSS feed from <a href="http://arxiv.org/">arXiv</a> is next to useless, it's so bloated. In amongst all that text, there must be connections and relationships that are being missed by the community at large, hidden in the chatter. Maybe we can develop simple techniques to highlight potential missed links, i.e. generate new hypotheses from the mass of words and data.<br />
<br />
<strong>Are the results coming out of the work useful for auditing academics?</strong><br />
<br />
<strong>Robert Simpson:</strong> Well, perhaps, but that would be tricky territory in my opinion. I've only just begun to explore the data around authorship in astronomy. One thing that is clear is that we can see a big trend toward collaborative work. In 2012, only 6% of papers were single-author efforts, compared with 70+% in the 1950s.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<img alt="The average number of authors per paper since 1827" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-7-avg-number-authors.png" style="margin-bottom: 15px;" width="450" />
<em>The above plot shows the average number of authors, per paper since 1827. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a>.</em></div>
</div>
We can measure how large groups are becoming, and who is part of which groups. In that sense, we can audit research groups, and maybe individual people. The big issue is keeping track of people through variations in their names and affiliations. Identifying authors is probably a solved problem if we look at <a href="http://about.orcid.org/">ORCID</a>.<br />
<br />
<strong>What about citations? Can you draw any comparisons with h-index data?</strong><br />
<br />
<strong>Robert Simpson:</strong> I haven't looked at h-index stuff specifically, at least not yet, but citations are fun. I looked at the trends surrounding the term "dark matter" and saw something interesting. Mentions of dark matter rise steadily after it first appears in the late '70s.
<br />
<div align="center">
<div style="border: 1px solid #ddd; font-style: italic; height: auto; margin: 15px 0 15px 0; padding: 10px; text-align: left; width: 450px;">
<a href="https://dl.dropbox.com/u/2068615/Orbiting%20Frog/sample-trends.html"><img alt="Compare the term dark matter with related terms" src="http://s.radar.oreilly.com/wp-files/2/2012/08/0812-8-dark-matter.png" style="margin-bottom: 15px;" width="450" /></a>
<em>Compare the term "dark matter" with a few other related terms: "cosmology," "big bang," "dark energy," and "wmap." You can see cosmology has been getting more popular since the 1990s, and dark energy is a recent addition. CREDIT: <a href="http://orbitingfrog.com/">Robert Simpson</a>.</em></div>
</div>
In the data, astronomy becomes more and more obsessed with dark matter — the term appears in 1% of all papers by the end of the '80s and 6% today.
Looking at citations changes the picture. The community is writing papers about dark matter more and more each year, but they are getting fewer citations than they used to (the peak for this was in the late '90s). These trends are normalised, so the only regency effect I can think of is that dark matter papers take more than 10 years to become citable. Either that or dark matter studies are currently in a trough for impact.<br />
<br />
<strong>Can you see where work is dropped by parts of the community and picked up again?</strong><br />
<br />
<strong>Robert Simpson:</strong> Not yet, but I see what you mean. I need to build a better picture of the community and its components.<br />
<br />
<strong>Can you build a social graph of astronomers out of this data? What about (academic) family trees?</strong><br />
<br />
<strong>Robert Simpson:</strong> Identifying unique authors is my next step, followed by creating fingerprints of individuals at a given point in time. When do people create their first-author papers, when do they have the most impact in their careers, stuff like that.<br />
<br />
<strong>What tools did you use? In hindsight, would you do it differently?</strong><br />
<br />
<strong>Robert Simpson:</strong> I'm using <a href="http://www.ruby-lang.org/en/" title="Ruby">Ruby</a> and <a href="http://www.perl.org/" title="Perl">Perl</a> to grab the data, <a href="http://www.mysql.com/" title="MySQL">MySQL</a> to store and query it, JavaScript to display it (<a href="https://developers.google.com/chart/" title="Google Charts">Google Charts</a> and <a href="http://d3js.org/" title="D3.js">D3.js</a>). I may still move the database part to <a href="http://www.mongodb.org/" title="MongoDB">MongoDB</a> because it was designed to store documents. Similarly, I may switch from <a href="http://adsabs.harvard.edu/abstract_service.html">ADS</a> to <a href="http://arxiv.org/">arXiv</a> as the data source. Using <a href="http://arxiv.org/">arXiv</a> would allow me to grab the full text in many cases, even if it does introduce a peer-review issue.<br />
<br />
<strong>What's next?</strong><br />
<br />
<strong>Robert Simpson:</strong> My aim is still to attempt real hypothesis generation. I've begun the process by investigating correlations between terms in the literature, but I think the power will be in being able to compare all terms with all terms and looking for the unexpected. Terms may correlate indirectly (via a third term, for example), so the entire corpus needs to be processed and optimised to make it work comprehensively.
<br />
<br />
<h2>
Science between the cracks</h2>
<br />
I'm really looking forward to seeing more results coming out of Robert's work. This sort of analysis hasn't really been possible before. It's showing a lot of promise both from a sociological angle, with the ability to do research into how science is done and how that has changed, but also ultimately as a hypothesis engine — something that can generate new science in and of itself. This is just a hack day experiment. Imagine what could be done if the literature were more open and this sort of analysis could be done across fields?<br />
<br />
Right now, a lot of the most interesting science is being done in the cracks between disciplines, but the hardest part of that sort of work is often trying to understand the literature of the discipline that isn't your own. Robert's project offers a lot of hope that this may soon become easier.Anonymoushttp://www.blogger.com/profile/01200256129780465367noreply@blogger.com2