tag:blogger.com,1999:blog-73883846276192737082024-03-13T00:13:28.225-07:00Web Visualization @ NCSUBen Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.comBlogger1237125tag:blogger.com,1999:blog-7388384627619273708.post-26865260040797760652013-12-11T17:44:00.001-08:002013-12-11T17:44:43.366-08:00Find: data and visualization politics begin #finds #visualization #data #politics<font color="#262626"><span style="font-size: 16px; line-height: 24px;">An early example of what will soon be a flood of data rhetoric. </span></font><br> <br> <div class="article"> <div class="source">// published on Latest Posts | The Atlantic Cities // <a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/FYbhEoYeMfw/story01.htm">visit site</a> </div> <div class="header"> <b>Why New Yorkers Are Hating on the NYPD's New Interactive Crime Map</b> </div> <div> <p> The New York Police Department <a href="http://www.nyc.gov/html/nypd/html/pr/pr_2013_12_08_nyc_crime_map.shtml">unveiled</a> a user-friendly interactive crime map last weekend. You can look at crime by precinct, by address, or on a heat map. One can also toggle between murder, rape, or a view that shows all seven major felony crimes.</p> <p> <img alt="" src="cid:(null)"></p> <p> The NYPD designed the interactive map at the behest of the New York City Council, and it's pretty good from a visual perspective. Yet New Yorkers aren't happy with it. </p> <p> Bronx Councilman Fernando Cabrera <a href="http://www.dnainfo.com/new-york/20131209/civic-center/nypds-crime-map-doesnt-do-city-justice-councilman-says">told <em>DNAInfo</em></a> that the map isn't at all what the council was talking about in May, when it mandated the NYPD to create a crime map. As you can see in the screenshot above, the NYPD's map doesn't contain important information like time and date, nor does it break out each incident with specific details. Cabrera added that the NYPD worked on the map "in obscurity," didn't keep the council updated on its progress, and ultimately has created a map that is "sub-standard to what you find in other states and in other cities, likes Chicago." </p> <p> The other major criticism of the map comes from advocates for bicycle and pedestrian safety. The <em>Village Voice</em> <a href="http://blogs.villagevoice.com/runninscared/2013/12/nypd_crime_map.php">argues</a> that the absence from the map of vehicular injuries and homicides is "a giant, gaping blind spot in the data visualization." It's also in keeping with NYPD Commissioner Ray Kelly's views on traffic deaths. </p> <p> As Sarah Goodyear <a href="http://www.theatlanticcities.com/commute/2013/10/what-role-should-police-play-preventing-traffic-deaths/7173/">reported</a> in October, Kelly believes prosecuting reckless driving is too complex. "Some people say that the police are not arresting enough people for reckless driving and that sort of thing. Well, you have to — and there are many court decisions that say this — you have to observe the violation," Kelly said at <em>The Atlantic</em>'s CityLab summit. "It takes in-depth investigation and examination, it takes witnesses, it's much more complex than you might think."</p> <p> If the NYPD omitted vehicular homicides from its crime map because it doesn't have the resources to investigate all vehicular homicides as crimes, why not do the next obvious thing, and create a separate map? The city council held a hearing in October to discuss this very idea. The NYPD's position? <em>We don't wanna</em>. As <em>Streetsblog</em> <a href="http://www.streetsblog.org/2013/10/10/nypd-public-too-stupid-to-understand-a-citywide-crash-map/">noted</a> at the time, Susan Petito, assistant commissioner of intergovernmental affairs for the NYPD, testified before the council that because vehicular accidents are cataloged by the nearest intersection, not the nearest address, the maps won't be useful or even all that intelligible.</p> <p> When asked if the NYPD would like to join the council in finding a way to map vehicle incidents more accurately, Petito replied, "The utility of a street address, I can't sit here and tell you that would add anything."</p> <p> Bicyclists and pedestrians just can't win with the current NYPD. </p></div></div> Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-55005202685721558952013-11-30T17:14:00.001-08:002013-11-30T17:14:42.835-08:00Data: Vaccinations have prevented at least 103 million cases of contagious disease since 1924 #data<br> <br> // published on The Verge - All Posts // <a href="http://www.theverge.com/2013/11/30/5160490/vaccines-prevent-at-least-103-million-cases-disease-since-1924">visit site</a><div class="article"> <div class="header"> <b>Vaccinations have prevented at least 103 million cases of contagious disease since 1924</b> </div> <div> <img alt="" src="http://cdn2.sbnation.com/entry_photo_images/9502671/vaccine-main-use_large.png"> <p>Vaccinations have been credited with some of humanity's greatest health technological triumphs over disease, including drastically <a href="http://www.who.int/mediacentre/factsheets/fs114/en/">reducing polio</a> around the globe and almost <a target="_blank" href="http://www.bt.cdc.gov/agent/smallpox/overview/disease-facts.asp">eliminating smallpox entirely</a>. But how many people have been spared life-threatening infections thanks to the introduction of vaccines? At least 103.1 million children in the US alone since 1924, according to a <a target="_blank" href="http://www.nejm.org/doi/full/10.1056/NEJMms1215400">new analysis</a> of historical infection rate data going back to 1888.</p> <p></p> <p> <a href="http://www.theverge.com/2013/11/30/5160490/vaccines-prevent-at-least-103-million-cases-disease-since-1924">Continue reading…</a></p></div></div> Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-28318857931251029852013-11-19T18:08:00.001-08:002013-11-19T18:08:33.781-08:00Find: A Bewitching Look at Migration Patterns Among American States #finds #census #visualizations<div><font color="#262626"><span style="font-size: 16px; line-height: 24px;">The color, curvature and symmetry here is especially appealing. However, the form doesn't scale well: it's hard to see the flows of all states at once, and the bidirectional nature of the flows aren't represented. A centered matrix would communicate</span></font><span style="font-size: 16px; line-height: 24px; color: rgb(38, 38, 38);"> more clearly, though some pleasing curvature would be gone. </span></div><div><br></div><div>Good inspiration for the nc innovation data. County to county flows? </div><div><br></div><div>*** </div><div> <div class="article"> <div class="source">// published on Latest Posts | The Atlantic Cities // <a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/VVSqHqGCYrM/story01.htm">visit site</a> </div> <div class="header"> <b>A Bewitching Look at Migration Patterns Among American States</b> </div> <div> <p> <a href="http://vizynary.com/2013/11/18/restless-america-state-to-state-migration-in-2012/"><img alt="" src="cid:(null)"></a></p> <p> New York residents must really get sick of the winter snow and gloom. How else to explain that more of them moved to Florida in 2012 than any other state?</p> <p> That's just one of the fascinating nuggets of <a href="http://www.theatlanticcities.com/topics/demographics/">demographic</a> trivia waiting to be uncovered in this wild-looking visualization of state-to-state migrations. The prismatic, arc-veined portal – like peering into the scope of an alien hyper-rifle – shows the movements of the roughly 7.1 million Americans who relocated across state lines in 2012. It's based on the U.S. Census Bureau's <a href="http://www.census.gov/hhes/migration/data/acs/state-to-state.html">American Community Survey</a>, an annual tabulation of moves that just so happens to include the involuntary uprootings of prisoners and members of the military.</p> <p> "<a href="http://vizynary.com/2013/11/18/restless-america-state-to-state-migration-in-2012/">Restless America</a>" is the work of Chris Walker, a data-analytics virtuoso in Mumbai who also made that clever visualization of <a href="http://www.theatlanticcities.com/housing/2013/11/surprising-map-new-yorks-two-cities/7480/">property values in New York City</a>. As to why he embarked on this project, Walker explains via email:</p> <blockquote> <p> I'm really interested in migration, as I think migration patterns show that people still see opportunity and hope for better lives, and they're willing to take risks. I see migration as a form of 'creative destruction'; it renews and enriches some communities while eroding others. This process strains individual cities, but I think it's healthy for the country overall. People need to dream and be allowed to act on their dreams. I wanted to show this on a national scale.</p> </blockquote> <div> The graphic may look like spaghetti pie at first glance, but it really is beautifully simple once you learn how to navigate it. Here's Walker explaining about that:</div> <blockquote> <p> The visualization is a circle cut up into arcs, the light-colored pieces along the edge of the circle, each one representing a state. The arcs are connected to each other by links, and each link represents the flow of people between two states. States with longer arcs exchange people with more states (California and New York, for example, have larger arcs). Links are thicker when there are relatively more people moving between two states. The color of each link is determined by the state that contributes the most migrants, so for example, the link between California and Texas is blue rather than orange, because California sent over 62,000 people to Texas, while Texas only sent about 43,000 people to California. Note that, to keep the graphic clean, I only drew a link between two states if they exchanged at least 10,000 people.</p> </blockquote> <p> For an example, let's go back to New York. If you put the mouse pointer over the state name, the graphic quickly informs you that more people recently exited than entered – 405,864 to 270,053, respectively. It also resolves into this minimalist view:</p> <p> <img alt="" src="cid:(null)"></p> <p> Gray strings represent all the states that New York sent more than 10,000 people to in 2012. The thickest band runs to Florida; click on it and you'll see that 53,009 New Yorkers headed for the Sunshine State and are perhaps appearing in <a href="https://twitter.com/_FloridaMan">Florida Man</a>'s Twitter feed this very instant. Conversely, 27,392 Floridians moved to New York and might now be experiencing <a href="http://www.theawl.com/2013/07/what-a-pack-of-cigarettes-costs-now-state-by-state">the joy</a> of $14.50 packs of cigarettes.</p> <p> Regarding the uneven transfer of bodies between these particular states, Walker writes that his "hunch is that these are retirees" decamping for the balmy Southeast. Other popular destinations for people escaping from New York include New Jersey and Pennsylvania, which Walker has a theory for, as well: "More likely these folks are leaving pricey New York City for more affordable suburbs in neighboring states."</p> <p> Not all interstate transmissions are this lucid. Take a look at California, for instance, which last year had migration pathways to more than 30 states:</p> <p> <img alt="" src="http://cdn.theatlanticcities.com/img/upload/2013/11/18/cali.JPG"></p> <p> With such a geyser of colored lines, it might be hard to immediately fathom a most basic point that in 2012 more people left California (566,986) than entered (493,641). Walker believes the imbalance may be due to residents tired of exorbitant prices seeking a lower cost of living. Here are a few more of his insights:</p> <blockquote> <ul> <li> Migrants are flocking to Florida. Interestingly the state contributing the most migrants to Florida is neighboring Georgia. Texas, New York, and North Carolina are the next largest contributors.</li> <li> Texas is the second-largest destination for migrants. Over 500,000 people moved to Texas in 2012. People tend to come from the Southeast, Southwest, and the West, with the biggest contributor being California. 62,702 Californians packed up and moved to the Lone Star state in 2012.</li> <li> Most people leaving DC tend to stay in the area, opting for Virginia or Maryland. The economy of DC, centered around the federal government, seems to discourage more distant migrations.</li> <li> The migrants who leave two very cold states, Maine and Alaska, have very clear preferences. Their most popular destinations are Florida and California.</li> </ul> </blockquote> <p> <em>Images created by <a href="http://vizynary.com">Chris Walker</a></em></p></div></div> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-43739566672952676732013-08-23T19:20:00.001-07:002013-08-23T19:20:05.010-07:00Find: visualizing how foods "go" together #finds #graphs #visualization <font color="#262626"><span style="font-size: 16px; line-height: 24px;">Not sure nodes and links were the best choice here. </span></font><div><font color="#262626"><span style="font-size: 16px; line-height: 24px;"><br></span></font></div><div><font color="#262626"><span style="font-size: 16px; line-height: 24px;">---<br></span></font><div><font color="#262626"><span style="font-size: 16px; line-height: 24px;"><br></span></font><div class="article"> <div class="source">// published on The Verge - All Posts // <a href="http://www.theverge.com/2013/8/23/4650706/pb-j-one-chart-shows-how-381-different-foods-will-taste-together">visit site</a> </div><div class="source"><br></div> <div class="header"> <b>PB&J: One chart shows how 381 different foods will taste together</b> </div><div class="header"><b><br></b></div> <div> <img src="http://cdn2.sbnation.com/entry_photo_images/8781477/foodchart_large.jpg" alt=""> <p>Some tastes just fit together perfectly — but why? This month, <a target="new" href="http://www.scientificamerican.com/article.cfm?id=flavor-connection-taste-map-interactive">Scientific American tackles the question</a> with an interactive chart, combining chemical analysis of 381 ingredients with data from over 50,000 recipes. The red lines indicate a shared chemical compound, like the common sugars between an apple and a glass of white wine. The analysis also unearths less expected links, like a surprising number of shared compounds between soybeans and black tea. It's a matter of taste whether those common compounds actually make the foods taste better together, but there's reason to think they do. <a target="new" href="http://www.nature.com/srep/2011/111215/srep00196/full/srep00196.html">The underlying research</a> finds that, in European cuisine at least, chefs tend to pair flavors based on shared chemistry. </p><p> <a href="http://www.theverge.com/2013/8/23/4650706/pb-j-one-chart-shows-how-381-different-foods-will-taste-together">Continue reading…</a></p></div></div> </div></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-24063676101412533252013-08-17T07:05:00.001-07:002013-08-17T07:05:22.176-07:00Find: New York City's Wealth Gap, Mapped In 3D<div><font color="#262626"><span style="font-size: 16px; line-height: 24px;">Mapping wealth to height feels intuitively right. Why? </span></font></div><div><font color="#262626"><span style="font-size: 16px; line-height: 24px;"><br></span></font></div><div><font color="#262626"><span style="font-size: 16px; line-height: 24px;">---<br></span></font> </div><div>// published on Latest Posts | The Atlantic Cities // <a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/vYM-kqSn5N0/story01.htm">visit site</a></div><div><br><div class="article"> <div class="header"> <b>Visualization of the Day: New York City's Wealth Gap, Mapped In 3D</b> </div> <div> <p> In a new set of visualizations, Nickolay Lamm, the artist and researcher who made these eerie <a href="http://www.theatlanticcities.com/arts-and-lifestyle/2013/07/apocalyptic-gif-art-day-san-francisco-under-25-feet-water/6220/">GIFs </a>of U.S. cities underwater, is drawing attention to the extensive of wealth inequality in Manhattan. </p><p> Lamm's inspiration came from standing atop Mt. Washington in his hometown of Pittsburgh and asking: what if you could see the inequality hidden behind the relatively even Pittsburgh skyline? Lamm explained via email why he ultimately decided to explore New York City's wealth gap instead: </p><blockquote> <p> I know that, for many people, moving New York City is the start of their journey to achieve the American Dream. The American Dream suggests that if you work hard enough, you can achieve it. However, it's clear that the landscape in order to achieve that dream is not as even and equal as it appears on the surface. </p></blockquote> <p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/1.jpg" alt=""><br> <em><small>View of Manhattan </small></em> </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/5.jpg" alt=""><br> <em><small>View from Central Park to Harlem </small></em> </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/4.jpg" alt=""><br> <em><small>View from Harlem </small></em> </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/3.jpg" alt=""><br> <em><small>View from Yankee Stadium in the Bronx </small></em> </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/2.jpg" alt=""><br> <em><small>View of Lower Manhattan </small></em> </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/08/16/map.JPG" alt=""> </p><p> Using the above <a href="http://www.arcgis.com/home/webmap/viewer.html?useExisting=1&services=814b785f25e24c9d8fc1a61ea61c0462">map</a> of median net worth in New York City (based on 2010 Census data), Lamm created 3D bar shapes for each block. So a block where the median income is $500,000 translates to a 5 cenitmeter bar, $112,000 to 1.12 centimeter, and so forth. Lamm then used Google Earth to make sure the bars were placed accurately on the photos. </p><p> What about other cities? Lamm says he may do Los Angeles, Chicago, Washington D.C., Miami, and or Philadelphia next.</p> </div> <br> <div class="visit"> <a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/vYM-kqSn5N0/story01.htm">Visit website</a></div></div> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-3106265287653424172013-06-19T18:32:00.001-07:002013-06-19T18:32:13.845-07:00Find: Map the iPhone Users In Any City, And You Know Where the Rich Live <div>Echoes of the recent pew survey showing that android is used more heavily among lower income groups. And if incomes group by both space and os, then os also clusters in space. </div><div><br></div><div>Ssup in Indonesia though? Maybe blackberry's security is important to them?? </div><div><br></div><div>---</div><div><br><div class="article"> <div class="source">// published on Latest Posts | The Atlantic Cities // <a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/9afOuVGST7c/story01.htm">visit site</a> </div> <div class="header"><b><br></b></div><div class="header"> <b>Map the iPhone Users In Any City, And You Know Where the Rich Live</b> </div> <div> <p> Our stuff often says a lot about us, whether we own a hybrid car or a station wagon, a MacBook Pro or an ancient desktop. And this is no less true of our smart phones, sold on a sharply divided market between iPhones, Androids, and Blackberries. </p><p> Among other things, cell phone brands say something about socio-economics – it takes a lot of money to buy a new iPhone 5 (and even <em>more</em> money to keep up with the latest models that come out faster than plan upgrades do). Consider, then, this map of Washington, D.C., which uses geolocated tweets, and the cell phone metadata attached to them, to illustrate who in town is using iPhones (red dots) and who's using Androids (green dots): </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/washington.png" alt="" style="width:605px;height:457px"> </p><p> That picture comes from a new series of <a href="http://www.mapbox.com/blog/visualizing-3-billion-tweets/">navigable maps</a> visualizing some three billion global, geotagged tweets sent since September of 2011, developed by <a href="http://gnip.com/">Gnip</a>, <a href="http://www.mapbox.com">MapBox</a> and dataviz guru <a href="http://www.theatlanticcities.com/arts-and-lifestyle/2012/08/mapmaker-artist-or-programmer/3132/">Eric Fischer</a>.* They've converted all of that data from the Twitter firehose (this is just a small fraction of all tweets, most of which have no geolocation data) into a series of maps illustrating worldwide patterns in <a href="http://www.mapbox.com/labs/twitter-gnip/languages/#5/48.239/5.449">language</a> and <a href="http://www.mapbox.com/labs/twitter-gnip/brands/#5/38.000/-95.000">device</a> use, as well as between people who appear to be tourists and locals in any given city. </p><p> The <a href="http://www.mapbox.com/labs/twitter-gnip/locals/#10/40.7591/-73.9852">locals and tourists</a> map scales up a beautiful <a href="http://www.flickr.com/photos/walkingsf/sets/72157624209158632/">earlier project</a> from Fischer. You could kill a few hours playing with all of these tools, built on the same dataset. But we particularly liked looking at the geography of smart phone devices. As in Washington, above, iPhones are often more prominent in upper-income parts of cities (and central business districts), while Androids appear to be the dominant device in lower-income areas. </p><p> These maps are also a blank canvas with nothing on them other than tweets. To the extent that you can easily make out the Washington Beltway above, or plenty of other roadway networks throughout the rest of these maps, that means people are tweeting while driving (or, preferably, sitting in the passenger seat). </p><p> Here is New York City, which has a smattering of Blackberries in Manhattan (yes, it's possible to <a href="https://twitter.com/download/blackberry">tweet from a Blackberry</a>). That green patch to the left is Newark: </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/new%20york%20city_.png" alt="" style="width:605px;height:440px"> </p><p> Here is Chicago: </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/chicago.png" alt=""> </p><p> And Houston: </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/houston.png" alt="" style="width:605px;height:436px"> </p><p> Atlanta: </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/atlanta.png" alt="" style="width:605px;height:506px"> </p><p> Los Angeles: </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/los%20angeles.png" alt="" style="width:605px;height:445px"> </p><p> And one place that <em>really</em> loves Blackberries? Jakarta. </p><p> <img src="http://cdn.theatlanticcities.com/img/upload/2013/06/19/jakarta.png" alt="" style="width:605px;height:444px"> </p><p> <em>Correction: This article initially misspelled Eric Fischer's name.</em> </p><p> <em>All images courtesy of MapBox, Gnip and Eric Fischer.</em></p><div><table border="0"><tbody><tr><td valign="middle"></td><td valign="middle"><a href="http://feedproxy.google.com/~r/TheAtlanticCities/~3/9afOuVGST7c/story01.htm">Visit website</a></td></tr></tbody></table></div></div></div> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-89838050771848508962013-03-14T05:00:00.001-07:002013-03-14T05:00:20.856-07:00Data: Public Data Website to Launch This Week<div class='posterous_autopost'><h4><a href="http://www.raleighpublicrecord.org/news/2013/03/13/public-data-website-to-launch-this-week/">Public Data Website to Launch This Week</a></h4><p>Data geeks can rejoice as the City of Raleigh makes more public information available through its new website.</p><br /><p>This week the City will launch Open Raleigh, a new platform for storing, accessing and visualizing public data. It’s appropriate timing, considering March 10-16 is <a href="http://www.sunshineweek.org/" target="_blank">Sunshine Week</a>, a celebration of open government.</p><br /><p>City officials hired <a href="http://www.raleighpublicrecord.org/www.socrata.com" target="_blank">Socrata</a>, a company that specializes in open data, to host the site, which will include various forms of public information. The city will spend $10,000 on a four-month pilot and then $50,000 for a full year after the pilot is over.</p><br /><p>When Open Raleigh launches in a beta version Friday, residents will have access to fire, police, census and building permit data as well as some financial data. More data will continue to be added as the public requests it.</p><br /><p>The data will be in a readable and downloadable format so it can be manipulated or visualized in a number of means. For example, neighborhood leaders can create crime maps for their communities, or mobile application developers can create an app that tells users where to find the nearest greenway.</p><br /><p>A full launch will be done in late September and will include a full financial ledger.</p><br /><p>Eventually, Open Raleigh will be compatible with other regional municipalities and include data from county, state and the federal government.</p><br /><p>City IT staff and members of the Technology and Communications Committee this week discussed ways the City could measure the return on investment and track how the data is being used, but more work will need to be done before that is possible.</p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-24110032437324537382013-01-01T06:33:00.001-08:002013-01-01T06:33:04.452-08:00Viz: The Census Dotmap contains one black dot for every person in the US, and nothing else<div class='posterous_autopost'><h4><a href="http://www.theverge.com/2012/12/29/3814720/the-census-dotmap-one-dot-for-every-person-living-in-us">The Census Dotmap contains one black dot for every person in the US, and nothing else</a></h4> <img src="http://cdn3.sbnation.com/entry_photo_images/7390539/Screen_Shot_2012-12-29_at_12.03.01_PM_large.png" height="420" alt="Census Dotmap" width="630" /><br /> <p>There are 308,450,225 dots on the map you see above. One for every person living in the United States, based on data from the 2010 Census. It doesn't seem plausible when you're looking at the entire country; metropolitan areas seem to be represented by indistinct swaths of black rather than tiny dots. But zoom in and even the most populated cities are finely broken down. There are no state borders, city lines, or individual roads represented on the Census Dotmap, as creator Brandon Martin-Anderson calls it. Nor is there any topographic data integrated here, yet the exhaustive reference still manages to offer a good sense of areas where nature prevents settlement. In fact, it lines up wonderfully with stunning <a href="http://www.theverge.com/2012/12/6/3734944/nasa-earth-night-photos" target="_blank">imagery of Earth at night...</a></p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-38784141902378193202012-11-28T17:10:00.001-08:002012-11-28T17:10:50.886-08:00Spotted: how differences in cognition and personality affect visualization - Understanding Visualization by Understanding Individual Users,<div class='posterous_autopost'><h4><a href="http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6353432">Understanding Visualization by Understanding Individual Users</a></h4> Visualization is often seen as a tool to support complex thinking. Although different people can have very different ways of approaching the kind of complex task that visualizations support, as researchers and designers we still rarely consider individual differences in creating and evaluating visualizations. This article reviews recent research on individual differences in visualization and human-computer interaction, showing that both cognitive abilities and personality profiles might significantly affect performance with these tools. The study of individual differences has led to the conclusion that advances in this important area in visualization will require more focused research. Specifically, we must isolate the cognitive factors that are relevant to visualization and the design factors that make one visualization more suited to a user than another. In doing so, we could increase our understanding of the visualization user and reshape how we approach design and evaluation.</div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-60719460102199338702012-11-27T17:06:00.001-08:002012-11-27T17:06:01.018-08:00Find: Beautiful and meaningful - Watch the beauty of public transit unfold in New York, Boston, and other cities around the world<div class='posterous_autopost'><h4><a href="http://www.theverge.com/2012/11/27/3699300/public-transit-visualization">Watch the beauty of public transit unfold in New York, Boston, and other cities around the world</a></h4> <img src="http://cdn2.sbnation.com/entry_photo_images/7278117/traffic_large.jpg" height="420" alt="nyc traffic video" width="630" /><br /> <p>YouTube user STLTransit has a wonderful collection of videos that visualize the movements of the public transit systems of various locations — turning public transit data into a flowing display of a city's daily life. Each video shows the ebb and flow of traffic during a single day, with various transit systems like bus and rail shown in different colors. The result is a brief but exciting glimpse at our world — a satellite's view of a city's living protoplasm on the streets below.</p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-57932441870573536862012-11-26T18:03:00.001-08:002012-11-26T18:03:14.001-08:00Find: Robert Kosara's visualization history<div class='posterous_autopost'> <table style="margin: 0px auto; padding: 0px; border-top-width: 1px; border-top-style: solid; border-top-color: rgb(187,187,187);"><tr><td style="border-top: 1px solid #bbb; padding: 20px 0; margin: 0;"> <h2 style="font-size: 15px; margin: 3px 0 0 0px; padding: 0; line-height: 1.5em;">Robert Kosara <span style="font-weight: normal;">(<a href="https://twitter.com/eagereyes" style="color: #777; text-decoration: none;">@eagereyes</a>)</span></h2> <p style="margin: 0 0 0 0px; font-size: 13px;"><a href="https://twitter.com/eagereyes/status/273099871291060224" style="color: #777; text-decoration: none;">11/26/12, 11:24 AM</a></p><p></p><p style="font-size: 15px; margin: 0 0 0 0px;"> New on eagereyes: The Changing Goals of Data Visualization <a href="http://t.co/zKorLpvJ" style="color: #286fb4;">eagereyes.org/criticism/chan…</a></p></td></tr></table> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com6tag:blogger.com,1999:blog-7388384627619273708.post-27896456065523745562012-11-12T17:56:00.001-08:002012-11-12T17:56:33.074-08:00Spotted: how people choose summarizing phrases - Descriptive keyphrases for text visualization<div class='posterous_autopost'><h4><a href="http://dl.acm.org/citation.cfm?id=2362367">“Without the clutter of unimportant words”: Descriptive keyphrases for text visualization</a></h4> Jason Chuang, Christopher D. Manning, Jeffrey Heer<p />Keyphrases aid the exploration of text collections by communicating salient aspects of documents and are often used to create effective visualizations of text. While prior work in HCI and visualization has proposed a variety of ways of presenting keyphrases, less attention has been paid to selecting the best descriptive terms. In this article, we investigate the statistical and linguistic properties of keyphrases chosen by human judges and determine which features are most predictive of high-quality descriptive phrases. Based on 5,611 responses from 69 graduate students describing a corpus of dissertation abstracts, we analyze characteristics of human-generated keyphrases, including phrase length, commonness, position, and part of speech.</div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-34216453361025755512012-10-18T20:42:00.001-07:002012-10-18T20:42:56.506-07:00Find: NYT Viz - How States Votes Have Shifted<div class='posterous_autopost'>Interesting visualization of bipolar values over time. Careful design here. <br /><h4><a href="http://fivethirtyeight.blogs.nytimes.com/2012/10/15/over-the-decades-how-states-have-shifted/">Over the Decades, How States Have Shifted</a></h4> An interactive graphic shows the political evolution of each state from 1952 through 2008 and the states' current standing in the FiveThirtyEight forecast.</div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com1tag:blogger.com,1999:blog-7388384627619273708.post-90179515681270737652012-09-21T10:07:00.001-07:002012-09-21T10:07:03.671-07:00Find: Build apps for elections with the Google Civic Information API<div class='posterous_autopost'><p> <a href="http://feedproxy.google.com/~r/GDBcode/~3/EciKk3mfJuI/build-apps-for-elections-with-google.html" style="display: block; color: #000; padding-bottom: 10px; text-decoration: none; font-size: 1em; font-weight: normal;"> <span style="display: block; color: #666; font-size: 1.0em; font-weight: normal;">Google Code Blog</span> <span style="font-size: 1.5em;">Build apps for elections with the Google Civic Information API</span> </a> </p> <img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_slHs0PsdpFerDImvpHVdGQcsT7Vl7fDNce9ysrNePsKF30VTgQ7ETLdGTvIZe60V8ZFRYz6jfzbR1AJGO5XUpN_22Q1LC-srUFnDXzvQQPIz1Yz1Fj7V_ZJcOFKOJ_ukpcdmraz-W-E/s1600/chetan.sabnis.jpg" height="80" alt="Author Photo" style="float: right; margin-left: 1em; text-align: right;" /><br /> <em>By Chetan Sabnis, Google Politics & Elections Team</em><p> <i>Cross-posted from the <a href="http://googlepolitics.blogspot.com/">Google Politics & Elections Blog</a></i><p> Google’s mission is to organize the world’s information and make it universally accessible and useful. To make it easier for you to build apps with voting information on the web, we are releasing our new <a href="https://developers.google.com/civic-information">Google Civic Information API</a>. The API enables you to look up comprehensive voting information for an individual U.S. address, including polling place, early vote sites, contests, and local election official contact information. By releasing this API, we hope to unleash the creativity of the Internet and help you build innovative products that push civic information to your communities in interesting ways. <p> As you know, this type of information can change frequently as we get closer to Election Day, and we will make every effort to ensure we're returning timely and accurate data. We have also included contact information for local election officials in the API so voters can find the most accurate information. <p> #more <p> While this API only includes voting information for elections in the U.S., we plan to expand to other countries and include other types of civic information. Please join the <a href="https://developers.google.com/civic-information/docs/ci_forum">Google Civic Information API Forum</a> for updates on the data available and check out the <a href="http://www.google.com/elections">Google Politics & Elections</a> page to find more information about the work we are doing around the election and our international elections programs.<br /> <div style="text-align: center;"><br /> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAa3kur3bnlOxgcANwe2Jool5yGswDcXUohwOAT5uU9mI3eXmv3hEpRo1ae1FYE1ZiUIDs8mvIxgJGT4c4WJqrwKnhH7FPaUXeja-BzymRCSuouUJ0LmsCfZFKxZ7cV3UDxBRievQVuek/s1600/election.png" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAa3kur3bnlOxgcANwe2Jool5yGswDcXUohwOAT5uU9mI3eXmv3hEpRo1ae1FYE1ZiUIDs8mvIxgJGT4c4WJqrwKnhH7FPaUXeja-BzymRCSuouUJ0LmsCfZFKxZ7cV3UDxBRievQVuek/s400/election.png" border="0" height="140" width="400" /></a></div> <br /> You can <a href="https://developers.google.com/apis-explorer/#p/civicinfo/us_v1/civicinfo.elections.voterInfoQuery?electionId=2000&_h=1&resource=%257B%250A++%2522address%2522%253A+%2522410+Market+St%252C+Chapel+Hill%252C+NC%2522%250A%257D&">get started here</a> through the Google APIs Explorer. The API is available now, but please note that full information isn't yet available for the November 6th General Election. We expect to be able to provide full live data around the middle of October, as it becomes available. For now, we recommend building your applications using the <a href="http://developers.google.com/civic-information/docs/us_v1/elections/electionQuery">test data</a> we provide. We'll be using the API to power our own election tools over the coming weeks, including an embeddable app anyone can use on their site, and we're looking forward to seeing the applications you come up with! <p> The Civic Information API replaces our previous <a href="http://developers.google.com/gec">Google Election Center API</a>, which will be turned down after January 1, 2013.<p> For questions, comments, and to showcase your apps using the API, we encourage you to use the <a href="https://developers.google.com/civic-information/docs/ci_forum">Google Civic Information API Forum</a>. <p> <br /> <em>Chetan Sabnis is a software engineer on the Google Politics & Elections team. In his spare time, Chetan enjoys solving logic puzzles and crosswords.</em><p> <em>Posted by <a href="https://profiles.google.com/u/0/105627346610764729807/about">Scott Knaster</a>, Editor</em></p></p></p></p></p></p></p></p></p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-20812260876251780002012-09-10T13:50:00.001-07:002012-09-10T13:50:35.688-07:00Viz: Facebook Freakonomics - a new tool to map the world's friendships<div class='posterous_autopost'><h4><span><a href="http://www.theverge.com/2012/9/10/3303178/facebook-freakonomics-mapping-the-worlds-friendships">Facebook Freakonomics: a new tool to map the world's friendships</a></span></h4> <img src="http://cdn1.sbnation.com/entry_photo_images/5376643/facebook_mapping_world_friendships_large.png" height="420" alt="facebook freakonomics mapping world's friendships" width="630" /><br /> <p>Last November, Facebook flipped Stanley Milgram's <a href="http://en.wikipedia.org/wiki/Six_degrees_of_separation" target="_blank">"six degrees of separation"</a> theory on its head and showed that <a href="http://www.theverge.com/2011/11/22/2580024/facebook-finds-that-4-74-degrees-of-separation-is-closer-to-the-truth" target="_blank">we're a lot closer than you might think</a>. Today, Facebook launched <a href="http://www.facebookstories.com/stories/1574/interactive-mapping-the-world-s-friendships" target="_blank">a new interactive map of the world's friendships</a> — a partnership with the data visualization wizards at the <a href="http://stamen.com" target="_blank">Stamen</a> design studio — that illustrates some of the ways we're connected. The map uses an often-amorphous blob of bubbles representing the world's countries to show which ones share the most friendships, and many times, even explain why.</p> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com3tag:blogger.com,1999:blog-7388384627619273708.post-61153117128028674822012-09-03T17:28:00.001-07:002012-09-03T17:28:08.738-07:00Viz: Amazon creates Presidential Election heat map based on political book purchases<div class='posterous_autopost'><h4><span><a href="http://www.theverge.com/2012/8/24/3266030/amazon-presidential-election-heat-map">Amazon creates Presidential Election heat map based on political book purchases</a></span></h4> <img src="http://cdn3.sbnation.com/entry_photo_images/5161053/amazonmap_large.jpg" height="420" alt="Amazon election heat map" width="630" /><br /> <p>Everyone from <a href="http://www.theverge.com/2012/8/23/3262549/microsoft-xbox-live-election-2012/in/2970698" target="_blank">Microsoft</a> to <a href="http://www.theverge.com/2012/8/1/3213129/twitter-political-index-romney-obama-tweet-comparison/in/2970698" target="_blank">Twitter</a> to <a href="http://www.theverge.com/2012/8/22/3260057/youtube-launches-elections-hub/in/2970698" target="_blank">YouTube</a> has taken to providing their own unique spin on covering the election, and Amazon is no different. The company has released a "2012 Election Heat Map" that shows what kinds of political books Americans are buying. Amazon took its 250 top-selling political books and characterized each one as either red (Republican Party), blue (Democratic party), or neutral, and then determined where those books have been purchased and shipped in the last 30 days. The result is a map that shows whether states are more red or blue based on their reading habits — at the time of writing Republican-leaning books were leading the way, accounting for 57 percent of purchases. Of course, as Amazon points out, "books...</p> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com1tag:blogger.com,1999:blog-7388384627619273708.post-31516976307144125032012-08-26T19:02:00.001-07:002012-08-26T19:02:21.033-07:00Viz: examining Wake county school superintendent Tata’s State of the Schools Speech with word clouds<div class='posterous_autopost'><h4><span><a href="http://www.raleighpublicrecord.org/news/2012/08/25/tatas-state-of-the-schools-speech-visualized/">Tata’s State of the Schools Speech, Visualized</a></span></h4> <p>If you thought school politics was difficult to understand before, check out this word cloud we created from the superintendent’s State of the Schools speech.</p><p>Tata’s speech Thursday focused mostly on school performance, as you can see with the prominence of the word “schools” in the word cloud. He also addressed the need for a school construction bond next year and stressed that new standardized tests, called the Common Core, might hit Wake with a reality check.</p> <p></p><center><a href="http://www-958.ibm.com/me/visualizations/combined-speech/comments/fc469370ee0a11e1aeb6000255111976" target="_blank" style="margin: 0pt; padding: 0pt;"> <img title="Combined Speech" src="http://www-958.ibm.com/me/files/thumbnails/fc27e6dc-ee0a-11e1-aeb6-000255111976.png?size=200x150" alt="Combined Speech" style="border: 1px solid #6898c8; margin: 0; padding-top: 10px; padding-bottom: 15px;" /> <img title="Many Eyes" src="http://www-958.ibm.com/me/images/blog_this_caption.jpg" alt="Many Eyes" style="border: 0pt none; margin: 0pt; padding: 0pt; display: block;" /></a></center> <p>Nerd note: We made the clouds using Many Eyes, by inputting the entire text of Tata’s speech. We then eliminated for redundancies like “schools” and “school.” We also eliminated words that didn’t seem to have much meaning like “get” and “year.”</p> <p><a name='more'></a></p><p>We had no idea what our word clouds would return when we started this project.</p><br /><p></p><center><a href="http://www-958.ibm.com/me/visualizations/tag-cloud-of-two-word-phrases/comments/bf093b9cee1011e1b0e3000255111976" target="_blank" style="margin: 0pt; padding: 0pt;"> <img title="Tag Cloud of Two Word Phrases" src="http://www-958.ibm.com/me/files/thumbnails/bed2c030-ee10-11e1-b0e3-000255111976.png?size=200x150" alt="Tag Cloud of Two Word Phrases" style="border: 1px solid #6898c8; margin: 0; padding-top: 10px; padding-bottom: 15px;" /> <img title="Many Eyes" src="http://www-958.ibm.com/me/images/blog_this_caption.jpg" alt="Many Eyes" style="border: 0pt none; margin: 0pt; padding: 0pt; display: block;" /></a><p> <em>This cloud shows the prominence of two-word phrases used during the speech.</em></p></center><p></p><br /><p>One thing to note is that both Tata and his chief of Family and Community Engagement, Cris Mulder, said the speech would not only address gains but the challenges the school system faces.</p> <br /><p>Tata did address some of the system’s challenges. When speaking of graduation rates he told the crowd, “what is painfully obvious here is that the state has essentially caught up with us.”</p><br /><p>However a word like “painfully” or “challenge” doesn’t show up with prominence in the word cloud, unlike “high” and “growth,” because Tata spent much time talking about school “gains.”</p> <br /><p>However, “NAEP” has a somewhat prominent place in the word cloud, because Tata stressed that North Carolina does not perform very well compared to other states.</p><br /><p>NAEP is the National Assessment of Educational Progress, a standardized test...</p> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-78991035971521125502012-08-19T19:18:00.001-07:002012-08-19T19:18:20.166-07:00Find: Novel text analysis uses PageRank to identify influential Victorian authors<div class='posterous_autopost'><h4><span><a href="http://feeds.arstechnica.com/~r/arstechnica/index/~3/XRsUJP66AC0/">Novel text analysis uses PageRank to identify influential Victorian authors</a></span></h4> <div style=""><br /> <img src="http://cdn.arstechnica.net/wp-content/uploads/2012/08/img154-2.jpg" height="413" width="620" /><br /></div> <p>A literature professor has developed software using Google's PageRank <a href="http://www.wired.co.uk/news/archive/2012-08/15/goldsmiths-motion-behaviour" target="_blank"> algorithm</a> that has identified Jane Austen and Walter Scott as the most influential authors of the 1800s.</p> <p>Matthew Jockers of the University of Nebraska analysed 3,592 digitized novels <a href="http://www.wired.co.uk/news/archive/2012-04/24/self-publishing-can-it-succeed" target="_blank"> published</a> in the UK, Ireland and the US between 1780 and 1900 using a combination of <a href="http://www.wired.co.uk/news/archive/2012-02/17/pagerank-for-hydrogen-bonds" target="_blank"> Google's algorithm</a>, <a href="http://www.wired.co.uk/news/archive/2011-12/19/song-prediction-algorithm" target="_blank"> machine learning</a> and a series of techniques used in computational text analysis including stylometry, corpus linguistics and network analysis.</p> <p>After ensuring the gender balance was split roughly evenly, Jockers went about using his software to extract thematic data&mdsah;this included the frequency of specific words or groups of words. Network software was then used to categorize and rank this data—Jockers began with a network consisting of 12,902,464 rows and three columns, with a source book allotted to the first column, a target <a href="http://www.wired.co.uk/news/archive/2012-08/17/book-stored-in-dna" target="_blank"> book</a> to the second and the third used to calculate the distance between the two (i.e. how many similarities they share according to the thematic data). After narrowing these data sets down to 6,447,640, the information was imported into network analysis software Gephi and PageRank was used to help identify down those novels which had the most links to future tomes, as well as the strongest links to those tomes.</p> <div><a href="http://feeds.arstechnica.com/~ff/arstechnica/index?a=XRsUJP66AC0:daJXj3tnNDs:V_sGLiPBpWU" target="_blank"></a></div></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com5tag:blogger.com,1999:blog-7388384627619273708.post-43754442515066557802012-08-16T18:36:00.001-07:002012-08-16T18:36:40.027-07:00Find: Twitter revs up mTurk with Clockwork Raven<div class='posterous_autopost'><div>Looks handy!</div><div><br /><h4><span><a href="http://engineering.twitter.com/2012/08/crowdsourced-data-analysis-with.html">Crowdsourced data</a> </span><span><a href="http://engineering.twitter.com/2012/08/crowdsourced-data-analysis-with.html">analysis with Clockwork Raven</a></span></h4> <p>Today, we’re excited to open source <a href="http://twitter.github.com/clockworkraven/" target="_blank">Clockwork Raven</a>, a web application that allows users to easily submit data to <a href="http://en.wikipedia.org/wiki/Amazon_Mechanical_Turk" target="_blank">Mechanical Turk</a> for manual review and then analyze that data. Clockwork Raven steps in to do what algorithms cannot: it sends your data analysis tasks to real people and gets fast, cheap and accurate results. We use Clockwork Raven to gather tens of thousands of judgments from Mechanical Turk users every week.</p> <h3>Motivation</h3><p>We’re huge fans of human evaluation at Twitter and how it can aid data analysis. In the past, we’ve used systems like Mechanical Turk and CrowdFlower, as well as an internal system where we train dedicated reviewers and have them come in to our offices. However, as we scale up our usage of human evaluation, we needed a better system. This is why we built Clockwork Raven and designed it with several important goals in mind:<p /> </p><div><a name='more'></a></div><p></p><h3>Features</h3><p>In Clockwork Raven, you create an evaluation by submitting a table of data (CSV or JSON). Each row of this table corresponds to a task that a human will complete. We build a template for the tasks in the Template Builder, then submit them to Mechanical Turk and Clockwork Raven tracks how many responses we’ve gotten. Once all the tasks are complete, we can import the results into Clockwork Raven where they’re presented in a configurable bar chart and can be exported to a number of data formats.</p> <p>Here’s the features we’ve built into Clockwork Raven to address the goals above:<p /></p><ul><li>Clockwork Raven has a simple drag-and-drop builder not unlike the form builder in Google Docs. We can create headers and text sections, add multiple-choice and free-response questions, and insert data from a column in the uploaded data. <p /> <br /></li></ul></div></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com1tag:blogger.com,1999:blog-7388384627619273708.post-74953841132977811812012-08-15T20:34:00.001-07:002012-08-15T20:34:20.110-07:00Viz: Distant Shape - 10 Years of Daring Fireball<div class='posterous_autopost'><a href="http://distantshape.com/df10/">http://distantshape.com/df10/</a> <p /> (Source: <a href="https://twitter.com/eagereyes/statuses/235826728809934848)">https://twitter.com/eagereyes/statuses/235826728809934848)</a></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com2tag:blogger.com,1999:blog-7388384627619273708.post-40931448577803531752012-08-15T19:18:00.001-07:002012-08-15T19:18:13.110-07:00Cassovary: A Big Graph-Processing Library<div class='posterous_autopost'><h4><span><a href="http://engineering.twitter.com/2012/03/cassovary-big-graph-processing-library.html">Cassovary: A Big Graph-Processing Library</a></span></h4> <p>We are open sourcing <a href="https://github.com/twitter/cassovary" target="_blank">Cassovary</a>, a big graph-processing library for the Java Virtual Machine (JVM) written in Scala. Cassovary is designed from the ground up to efficiently handle graphs with billions of edges. It comes with some common node and graph data structures and traversal algorithms. A typical usage is to do large-scale graph mining and analysis.</p> <br /><p>At Twitter, Cassovary forms the bottom layer of a stack that we use to power many of our graph-based features, including <a href="https://twitter.com/who_to_follow/suggestions" target="_blank">"Who to Follow"</a> and <a href="https://twitter.com/similar_to/twitter" target="_blank">“Similar to.”</a> We also use it for relevance in <a href="http://engineering.twitter.com/2011/05/engineering-behind-twitters-new-search.html" target="_blank">Twitter Search</a> and the algorithms that determine which Promoted Products users will see. Over time, we hope to bring more non-proprietary logic from some of those product features into Cassovary.</p> <br /><p>Please use, fork, and contribute to Cassovary if you can. If you have any questions, ask on the <a href="http://www.google.com/url?q=http%3A%2F%2Fgroups.google.com%2Fgroup%2Ftwitter-cassovary" target="_blank">mailing list</a> or file <a href="https://github.com/twitter/cassovary/issues" target="_blank">issues</a> on GitHub. Also, follow <a href="https://twitter.com/cassovary" target="_blank">@cassovary</a> for updates.</p> <br /><p>-Pankaj Gupta (@<a href="https://twitter.com/#!/pankaj" target="_blank">pankaj</a>)</p><div><img src="https://blogger.googleusercontent.com/tracker/5340805191653517637-2363946451110376071?l=engineering.twitter.com" height="1" alt="" width="1" /></div> </div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-3272384690151999342012-08-15T19:09:00.001-07:002012-08-15T19:09:18.497-07:00Viz: Visualizing Hadoop with HDFS-DU<div class='posterous_autopost'><h4><span><a href="http://engineering.twitter.com/2012/08/visualizing-hadoop-with-hdfs-du.html">Visualizing Hadoop with HDFS-DU</a></span></h4> <p>We are a heavy adopter of <a href="https://github.com/twitter/hdfs-du" target="_blank">Apache Hadoop</a> with a large set of data that resides in its clusters, so it’s important for us to understand how these resources are utilized. At our July <a href="http://blog.twitter.com/2012/01/hack-week-twitter.html" target="_blank">Hack Week</a>, we experimented with developing <a href="https://github.com/twitter/hdfs-du" target="_blank">HDFS-DU</a> to provide us an interactive visualization of the underlying Hadoop Distributed File System (HDFS). The project aims to monitor different snapshots for the entire HDFS system in an interactive way, showing the size of the folders and the rate at which the size changes. It can also effectively identify efficient and inefficient file storage and highlight nodes in the file system where this is happening.</p> <p>HDFS-DU provides the following in a web user interface:<br /></p><ul><li>A TreeMap visualization where each node is a folder in HDFS. The area of each node can be relative to the size or number of descendents</li></ul><br /><li> A tree visualization showing the topology of the file system</li><p>HDFS-DU is built using the following front-end technologies:<br /></p><ul><li><a href="http://d3js.org/" target="_blank">D3.js</a>: for tree visualization</li> </ul><br /><li><a href="http://thejit.org/" target="_blank">JavaScript InfoVis Toolkit</a>: for TreeMap visualization</li><h3>Details</h3><p>Below is a screenshot of the HDFS-DU user interface (directory names scrubbed). The user interface is made up of two linked visualizations. The left visualization is a TreeMap and shows parent-child relationships through containment. The right visualization is a tree layout, which displays two levels of depth from the current selected node in the file system. The tree visualization displays extra information for each node on hover.</p> <div style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTiMI-8itujNDBZ-3JxOhytY6OwNfOlt9un9KMlJcWAk5wbJrV6-f_GSx3vS-NDH0HafFZIkWj3teCq4bZHaOQyyyG6eDQ49wtRmNkaB0cwUIoHUAM1cFJA8EH4yXZZ6PR0ks12NMZ2Rj/s1600/1.png" target="_blank" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTiMI-8itujNDBZ-3JxOhytY6OwNfOlt9un9KMlJcWAk5wbJrV6-f_GSx3vS-NDH0HafFZIkWj3teCq4bZHaOQyyyG6eDQ49wtRmNkaB0cwUIoHUAM1cFJA8EH4yXZZ6PR0ks12NMZ2Rj/s400/1.png" border="0" height="352" width="400" /></a></div> <p>You can drill down on the TreeMap by clicking on a node, this would create the same effect as clicking on any tree node. There are two possible layouts for the TreeMap. The default one encodes file size in the area of each node. The second one encodes number of descendents in the area of each node. In the second view it's interesting to spot nodes where storage is inefficient.</p> <p><a name='more'></a></p><div style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1JbJkbOk3yA0W05uy6dTkFrGwLT8CH3vynk-FGmZb8KQL_Ya2e6LIKibF2yRm3HuRmEsFLttZ5rwZy5p1G2ZyrDMiqsZQjpbIEDhvVXFCqsre9oyS0ANosfDUHOrPUhy2wa2k2INipUBf/s1600/2.png" target="_blank" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1JbJkbOk3yA0W05uy6dTkFrGwLT8CH3vynk-FGmZb8KQL_Ya2e6LIKibF2yRm3HuRmEsFLttZ5rwZy5p1G2ZyrDMiqsZQjpbIEDhvVXFCqsre9oyS0ANosfDUHOrPUhy2wa2k2INipUBf/s400/2.png" border="0" height="309" width="400" /></a></div> <p><h3>Future Work</h3><p>This proj...</p></p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-28218673648457140512012-08-15T18:58:00.001-07:002012-08-15T18:58:50.410-07:00Find: visualizing the evolution of the web<div class='posterous_autopost'><h4><span><a href="http://feedproxy.google.com/~r/GDBcode/~3/L7Edla82lLc/behind-scenes-visualizing-evolution-of.html">Behind the scenes: visualizing the evolution of the web</a></span><br /> </h4><table style="float: right; margin-left: 1em; text-align: right;"><tr><td style="text-align: center;"><img src="http://2.bp.blogspot.com/--kA9oQYwlRw/UCRL5Xj2IQI/AAAAAAAAB00/0LbKLWq5FLo/s1600/deroy.peraza.png" height="80" style="margin-left: auto; margin-right: auto; text-align: right;" /></td> </tr><tr><td style="text-align: center;">Dero</td></tr></table><br /><table style="float: right; margin-left: 1em; text-align: right;"><tr><td style="text-align: center;"><img src="http://2.bp.blogspot.com/-MjX2PaUmafU/UCRL5JpjRjI/AAAAAAAAB0k/pId2reTv7NE/s1600/sergio.alvarez.jpg" height="80" style="margin-left: auto; margin-right: auto; text-align: right;" /></td> </tr><tr><td style="text-align: center;">Sergio</td></tr></table><p><em>This guest post is by Sergio Alvarez, <a href="http://vizzuality.com/" target="_blank">Vizzuality</a>, and Deroy Peraza, <a href="http://hyperakt.com/" target="_blank">Hyperakt</a>, in collaboration with Min Li Chan, Chrome Team</em><p> At <a href="http://googledevelopers.blogspot.com/2012/06/google-io-2012-thats-wrap.html" target="_blank">Google I/O</a> this year, we launched a new version of <a href="http://www.evolutionoftheweb.com/" target="_blank">The Evolution of the Web</a>, a project visualizing the history and pace of innovation in web technologies and browsers. <a href="http://www.evolutionoftheweb.com/" target="_blank">The Evolution of the Web</a> traces how web technologies have evolved in the last two decades and highlights the web community’s continuous efforts to improve the web platform and enable developers to create new generations of immersive web experiences. In collaboration with the <a href="http://chrome.blogspot.com/" target="_blank">Google Chrome</a> team, the team at <a href="http://www.hyperakt.com/" target="_blank">Hyperakt</a> designed the interactive visualization while <a href="http://vizzuality.com/" target="_blank">Vizzuality</a> built it using HTML5, SVG, and CSS3.<p> <p><div style="text-align: center;"><a href="http://3.bp.blogspot.com/-ZqeIQJUt320/UCRL4RFzaPI/AAAAAAAAB0A/riQPnVup18E/s1600/evolution.png" target="_blank" style="margin-left: 1em; margin-right: 1em;"><img src="http://3.bp.blogspot.com/-ZqeIQJUt320/UCRL4RFzaPI/AAAAAAAAB0A/riQPnVup18E/s400/evolution.png" border="0" height="150" width="400" /></a></div> <p>The visualization included 43 web technology "strands" across 7 browser timelines to represent major developments on the web platform. On hover or tap, each strand is highlighted to reveal intersections that tell the story of when browser support was implemented for each new web technology. To provide additional context, we developed a secondary visualization to illustrate the growth of Internet users and traffic. <p> <p><p /></p></p></p></p></p></p></p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com1tag:blogger.com,1999:blog-7388384627619273708.post-3145094377973483112012-08-12T11:27:00.001-07:002012-08-12T11:27:15.235-07:00Viz: Tracking athletes' Twitter mentions over the Olympics<div class='posterous_autopost'><h4><span><a href="http://www.theverge.com/2012/8/11/3236901/tracking-athletes-twitter-mentions-over-the-olympics">Tracking athletes' Twitter mentions over the Olympics</a></span></h4> <img src="http://cdn0.sbnation.com/entry_photo_images/5000979/s05he8u9m_ob_large.png" height="420" alt="New York Times Twitter olympics mentions infographic" width="630" /><p> <p><p><br /> <p><i>The New York Times</i> is back with <a href="http://www.theverge.com/2012/8/6/3222828/usain-bolt-new-york-times-visualization" target="_blank">another excellent infographic about the 2012 Olympics in London</a> — this time showing Twitter activity on athletes' accounts. <a href="http://www.nytimes.com/interactive/2012/08/09/sports/olympics/new-olympic-stars-of-twitter.html?hp" target="_blank">The graphic</a> visualizes the number of mentions 140 verified accounts received over the games so far per 1,000 followers, honing in on when different athletes' mindshare peaked on Twitter. So, who won? Malaysian track cyclist Azizulhasni Awang (@AzizulAWANG) looks to have received the the most mentions per 1,000 followers (2,308) after his <a href="http://twitter.ie/AzizulAWANG/status/233223701305360386" target="_blank">public apology</a> for failing to obtain any medals. Michael Phelps' mentions, meanwhile, were dwarfed by his over 1,000,000 follower count.</p> </p></p></p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0tag:blogger.com,1999:blog-7388384627619273708.post-33096795881504968182012-08-11T19:23:00.001-07:002012-08-11T19:23:28.781-07:00Find: Census Bureau releases 'America's Economy' app for Android<div class='posterous_autopost'>Good idea. <br /><h4><span><a href="http://www.theverge.com/2012/8/11/3234563/census-bureau-americas-economy-app-android">Statistics on the go: Census Bureau releases 'America's Economy' app for Android</a></span></h4> <img src="http://cdn3.sbnation.com/entry_photo_images/4989746/Americaseconomyapp_large.jpg" height="420" alt="America's Economy App" width="630" /><p> <span>The US Census Bureau just released its <a href="http://www.theverge.com/2012/7/30/3200199/us-census-bureau-releases-public-api" target="_blank">public API</a> last month, and now the agency has released the America's Economy app for Android phones and tablets with an iOS version in the works. By combining its own data with that of the Bureau of Economic Analysis and the Bureau of Labor Statistics, the Census Bureau's app will track real-time trends in "employment, housing construction, international trade, personal income, retail sales and manufacturing." The app was created as part of the Census Bureau's <a href="http://blogs.census.gov/2012/07/25/pardon-our-dust-census-gov-transformation/" target="_blank">Web Transformation Project</a> to help federal employees and the general public access accurate data about the US' socioeconomic health. Both tablet and phone versions of the app are available in the Google Play store now.</span><br /> </p></div>Ben Watsonhttp://www.blogger.com/profile/16654330374746823837noreply@blogger.com0