Visualizing data is like photography. Instead of starting with a blank canvas, you manipulate the lens used to present the data from a certain angle.
When the data is the social graph of 500 million people, there are a lot of lenses through which you can view it. One that piqued my curiosity was the locality of friendship. I was interested in seeing how geography and political borders affected where people lived relative to their friends. I wanted a visualization that would show which cities had a lot of friendships between them.
I began by taking a sample of about ten million pairs of friends from Apache Hive, our data warehouse. I combined that data with each user's current city and summed the number of friends between each pair of cities. Then I merged the data with the longitude and latitude of each city.
At that point, I began exploring it in R, an open-source statistics environment. As a sanity check, I plotted points at some of the latitude and longitude coordinates. To my relief, what I saw was roughly an outline of the world. Next I erased the dots and plotted lines between the points. After a few minutes of rendering, a big white blob appeared in the center of the map. Some of the outer edges of the blob vaguely resembled the continents, but it was clear that I had too much data to get interesting results just by drawing lines. I thought that making the lines semi-transparent would do the trick, but I quickly realized that my graphing environment couldn't handle enough shades of color for it to work the way I wanted.
Instead I found a way to simulate the effect I wanted. I defined weights for each pair of cities as a function of the Euclidean distance between them and the number of friends between them. Then I plotted lines between the pairs by weight, so that pairs of cities with the most friendships between them were drawn on top of the others. I used a color ramp from black to blue to white, with each line's color depending on its weight. I also transformed some of the lines to wrap around the image, rather than spanning more than halfway around the world.
After a few minutes of rendering, the new plot appeared, and I was a bit taken aback by what I saw. The blob had turned into a surprisingly detailed map of the world. Not only were continents visible, certain international borders were apparent as well. What really struck me, though, was knowing that the lines didn't represent coasts or rivers or political borders, but real human relationships. Each line might represent a friendship made while travelling, a family member abroad, or an old college friend pulled away by the various forces of life.
Later I replaced the lines with great circle arcs, which are the shortest routes between two points on the Earth. Because the Earth is a sphere, these are often not straight lines on the projection.
When I shared the image with others within Facebook, it resonated with many people. It's not just a pretty picture, it's a reaffirmation of the impact we have in connecting people, even across oceans and borders.
YOU ARE NOT A CAT
PLEASE STOP TAKING PICTURES LIKE THIS
WHY DO YOU DO THIS
STOP COVERING HALF YOUR FACE WITH YOUR HAND OR JUMPER
STOP BITING YOUR JUMPER
NO
STOP
IT
A;KFJAD;SLFKDJ
YOU ARE NOT A CAT
We look. NOTHING. alike. :L
This my little sister Ciara and myself yesterday before I went and spent the day with @CRTS once more.
I did more shopping, he tagged along, went to Starbucks, broke a few picture frames, y'know, the usual. Then came back to mine and played Cooking Mama on the Wii and ate pizza. OMNOMNOM.
It saddens me that I have to leave so soon. :/ I really wish I could stay just a few days longer.
If you haven't met @CRTS yet then your life is not complete, just sayin'. He really is one of the nicest, funniest guys I've ever met, and I'm totally comfortable around him. He is amazing and I know I'm gonna know him for a fucking long time to come. And I'm going to miss him and think about him every single day until I see him again. :) And all you girls who think he's like, SUPAH HAWT, etc....well you're totally right 'cuz he is super hawt and he has awesome hair. And he constantly does this "bieber hair flick" and everytime he does it I shout "BIEBER FLICK" and he gets all smiley and tries to restrain himself from doing it. You made my holiday, Curtis Radcliffe! I love yew.
p.s see you tomorrow, shmurtis.
Dunno what I'm doing today. Think I was meant to go to the cinema again but that plan went out the window. Hmmmm.
