RSS

Author Archives: visualign

About visualign

Visualize Data. Improve Performance.

Kayaking South-East Florida

Over the last few weekends I have gone out on my old Kayak and played around several Inlets. The weather pattern throughout several weeks in June 2014 was conducive for going out on the Ocean: Light West wind in the morning creates very calm water, and low tide in the late morning allows to first ride out the inlet with the outgoing tide, then play around on the open water or rest somewhere on the beach, and around noon or early afternoon ride the incoming tide back through the inlet. Plus on most days the Wind shifts to SouthEast for the familiar on-shore sea-breeze once the day gets really hot. So on an ideal day you can paddle with the current and tailwind twice on an out-&-back round-trip!

Jupiter Inlet

The first of those trips was on Sunday, Jun-8, at the Jupiter inlet. I parked the car near the Southern end of Jupiter Island, not far from the Coral Cove Park. One of the advantages here is that you can go out the Intracoastal, past the turn with the Jupiter lighthouse, then out the inlet. The water here can be very clear, but only when the tide is coming in – otherwise you get the murky water from the Loxahatchee River. Here is the GPS track from this outing:

JupiterInlet

Not surprisingly, it is a very hot day. The color of the graph shows temperature, generally between 27C (green) and about 37C (red). Once further out on the calm water I strap on the snorkel gear and go for a swim. During the 20-25min intervals the temperature is that of the Ocean water, a refreshing 27C (marked in green). Note how I drift during the snorkel with the Kayak due to the Gulf stream Ocean current, more significant the further out one goes. I tie a dive flag to the Kayak and myself, so I do get noticed in the water by boat traffic and do not get separated from the Kayak. I also use the Inlet as a way to orient myself, specifically the ability to see the A1A bridge through the Inlet.

P1120628

Launching on the Intracoastal just North of the Jupiter Island bridge

P1120635

Under the Bridge to Jupiter island

P1120631

Passing other Kayaks on the Intracoastal

P1120651

Taking a break on the jetty of the Jupiter Inlet, looking North to the beaches of Jupiter Island

P1120657

The Jupiter Lighthouse

P1120663

Drifting back in with incoming tide under Jupiter Island bridge

This trip was about 12 km and 3.5 hours. One of the most enjoyable aspects was the snorkel way out in the Ocean. I had never gone out that far off the coast, almost 3 km. My only concern was to watch for thunderstorm clouds forming, as you do not want to get caught far off the beach when a storm is brewing. The next day I read in the newspaper that about 2 hours after I stood there and took the photo at the jetty, a boating accident killed one young man snorkeling in the water right near the inlet. One always has to pay attention when in the water.

Palm Beach Inlet and Peanut Island

The following weekend (Sun, Jun-15) I take the kayak out to nearby Palm Beach inlet. I park at the Phil Foster Park, right on the East end of the big Blue Heron bridge. This is a popular dive and snorkel spot, with some artificial reefs and shallow water diving with lots of fish right off the parking lot. Here is the GPS track, again with temperature overlay.

PalmBeachInlet

I didn’t time it quite as well today, as the tide is already starting to come in as I’m starting and there is a good SE breeze and as a result a moderate 3-5 ft swell on the open water. Consequently I am hugging the North shoreline of the Inlet so as to not having to work too hard against the current of the incoming tide. This gives me a good upper-body workout as this outing overall will be a bit shorter. Once out of the inlet I cross the current and round a large buoy. There are too many waves and so I decide not to snorkel out here. Instead, I let the current carry me back in and then pause at a small pier on the Northern End of N Ocean Blvd on Palm Beach Island. I remember this place form many bike rides up from Delray Beach which naturally end on this pier. Under the pier there are some coral and quite a lot of fish, so it’s actually fairly interesting to snorkel around here. (See green spot in the GPS track due to the cooler water temperature.)

After that refreshing break I continue with the current and decide to round Peanut Island on the way back. There is a good amount of boat traffic on a sunny weekend so I need to pay attention and stay close to the shore. The trip was about 6.5 km and 2 hours.

Saint Lucie Inlet and Hutchinson Island

The following weekend (Sun, Jun-22) I want to go for a bit longer trip. I start early (6:30am) and drive about 45min North to the town of Stuart. The Sandsprit Park there is a popular place to launch boats and has generous facilities and parking space. By the time I get on the water it is about 8am. From my Bing map it appears as if I can do a loop around some canals, which I decide to try. Unfortunately this turns out to be a dead end after about 80% of the distance, so I have to return the same way and pass the launch site after 1.5 hrs of very hot paddling (due to lack of Ocean breeze). Here is the GPS track of the entire day:

StuartInlet

After that dead-end round-trip I enjoy the open water of the the Saint Lucie Inlet, separating Jupiter Island to the South form Hutchinson Island to the North. The tide is going out and the current is now very strong. Normally my speed on the old double kayak is about 5 km/h in calm water, on this stretch my average is about 9 km/h, with max above 11 km/h. I could not go back directly against such a strong current, which is why I need to bring plenty of time (to wait for the tide to turn) or find another way to return. I stop at a sandbank at the Northern tip of Jupiter Island. As there are no roads leading to this part of the island it is very quiet and peaceful here. After a little break I cross the inlet and round the tip of Hutchinson Island. The rocks of the jetty here are spaced apart and except for low tide one can pass through them. I need to pay attention here as the current of the tide around these rocks feels like a river with rocks in the middle. Beyond that to the North lies to so-called Bathtub Reef, a very nice spot for snorkeling. I spend about 1/2 hour in the water, which is very refreshing. I swim in the calm and shallow waters to see various coral formations.

Launching at Sandsprit Park

Launching at Sandsprit Park

One of several road bridges in the canal near Sandsprit Park

One of several road bridges in the canal near Sandsprit Park

Paddling out the Saint Lucie Inlet with Jupiter Island to the right

Paddling out the Saint Lucie Inlet with Jupiter Island to the right

Taking a break on the sand bank

Taking a break on the sand bank

Near Bathtub Beach at site of portage across the road to get to Intracoastal

Near Bathtub Beach at site of portage across the road to get to Intracoastal

After portage launching into Intracoastal for direct line back to Sandsprit Park

After portage launching into Intracoastal for direct line back to Sandsprit Park

Back at Sandsprit Park

Back at Sandsprit Park

One solution to the problem of going against the tide current is to avoid the inlet altogether: I go North and portage across the very narrow part of Hutchinson Island, essentially just across the beach and the road at Bathtub Beach. This brings me back to the wide Intracoastal. Due to the now very low tide there are some stretches of very shallow water; at times I have to get out and walk as there is barely more than a few inches of water under the kayak. Coming back around the Southern tip of Sewalls Point it gets very hot again, around 35C. I am fairly tired by the time I’m back at Sandsprit Park and happy to get out of the sun and heat. Good thing I brought enough sunscreen and applied it repeatedly to avoid sunburn. This was a much longer trip than the previous two, about 21 km and 4.5 hours.

Miami and Key Biscayne

After a few weeks of rainy and stormy weather there is finally another good weekend for kayaking in July. This time I plan to go back down to Miami and circumnavigate Key Biscayne, which I had done twice before, once with my son Philip many years ago. Paddling around Key Biscayne is about a 4 hour trip and provides a nice variety of sandy beaches, a lighthouse and mangrove settings. This time, however, I have an even longer loop in mind: How about combining Key Biscayne with the Miami downtown area and the canal for the big cruise ships? I check the tide schedule and with low tide at 9:45am I figure I need to start very early to take full advantage of the tide current.

I load up the kayak on my car the evening before so I can leave very early in the morning. At 5:30am on Saturday, Jul-19 I start driving down to Miami. Unfortunately there is over-night construction on I95 reducing the highway down to just one lane, which costs me about a half hour in stop-&-go traffic. Not what I had planned, but I am still early enough by the time I get to the Rickenbacker Causeway and park my car. From here I plan a roughly rectangular course. Here is the GPS track:

MiamiBiscayneBay

The first hour or so I’m following the skyline of downtown Miami. It’s somewhat surreal to be paddling right next to tall skyscrapers and hotel buildings. At the very beginning there are even a few raindrops, which makes me wonder whether the forecast of low percentage of rain is going to come to pass. I’m rounding the Brickell Key which opens up the view towards the Miami river. Big yachts are mooring here, right in between the downtown hotels and office buildings.

View at mouth of Miami River near Brickell Key

View at mouth of Miami River near Brickell Key

From here its just a short distance to Bayside, with its Hard Rock Cafe and various other cafes and restaurants. I remember this place from many visits a long time ago when I first moved to Florida, by now 18 years ago. I pass the American Airlines Arena, among others home to the Basketball games of the Miami Heat. Next up are the big cruise ships on Dodge Island, Carnival Cruise Lines and other megaships. I notice one of those cruise ships closing the back loading dock and think to myself, perhaps I’ll see this ship later on.

Cruise Ships in Miami

Cruise Ships in Miami

In the meantime I go a bit further North under the bridge of the MacArthur Causeway, the big highway connecting Miami with Miami Beach. Here I need to paddle fast against the noticeable tidal current pulling the water out the big ship channel to the Ocean.I round the NW corner of the rectangular course and head roughly East towards Miami Beach. There are a few of the small, artificial islands with some of the most expensive real estate in the world: Palm Island, Hibiscus Island and Star Island. I get a slight current pulling me along while I look at the mega-mansions along the water line. Here is a photo of a 28,000 sq.ft mega-estate on a 6 acre lot on Star Island, assessed at $52m for tax purposes according to Zillow:

Big Estate on Star Island

Big Estate on Star Island

So definitely a lot of variety here, with prime views from the waterline in the kayak. Then I round the Miami Beach marina and follow the government cut canal out to the Ocean. At one point a small Pilot boat passes me; this probably means there is some bigger vessel following behind. And when I turn around near the end of the rock jetty, as expected from earlier in my paddle, I see the Bimini Resort cruise ship coming by. Quite impressive the size of these things when you look at it from a kayak:

Cruise Ship leaving the Miami inlet channel

Cruise Ship leaving the Miami inlet channel

The water displacement effect of this ship in the narrow channel is quite noticeable. First the water is pushed out sideways which creates an outflowing current around the rocks – I need to paddle a bit to stay clear of the last rocks. Just a half minute later the current reverses as the water is rushing back into the channel once the big vessel has parted.

I decide to take a bit of a break here in the calm and clear water and go snorkel on the North side of the rocks lining the channel. The by now familiar change into snorkel gear – fins, mask and snorkel – and I’m floating in the water holding on to my trusted companion:

Snorkeling next to Kayak near Miami Ship channel outlet

Snorkeling next to Kayak near Miami Ship channel outlet

I apply some more sunscreen as the sun is getting really intense now. The following section from here on South is less eventful. Fisher Island and then Virginia Beach pass by slowly on the right side. My course is towards the first park on Key Biscayne. It is about 4 hours after I left in the morning by the time I reach the beach at Crandon Park. I need to get some rest, water and shade. I pull the kayak ashore, take my drybag and the (expensive) carbon paddles with me, then retreat to the shade of a big tree next to some shower and restroom facilities. I eat some snacks and refill one of the four water bottles I brought about 3-4 times. I will need to stay hydrated for such a long paddle.

Lunch break and rest at Crandon Park on Key Biscayne

Lunch break and rest at Crandon Park on Key Biscayne

After a bit more than 1 hour I feel refreshed and continue on south. Lots of water sports here, wave-runners and standup paddle boards, plus hundreds of people swimming or just lounging in the water to escape from the heat. In the meantime, there are some big storm clouds brewing just West of Miami. At times they get so big that even the big jets intending to land at Miami International airport fly a few big loops to wait for them to pass. I sure hope they will not come further East out to the water, as that would require me to pause, if not abandon my kayak trip.

Big storm clouds brewing West of Miami

Big storm clouds brewing West of Miami

About 6.5 hours after I started in the morning I finally get close to the Cape Florida Lighthouse marking the Southern tip of Key Biscayne. It is a beautiful summer day and there are big crowds here in the Bill Baggs Cape Florida State Park. No entrance fee when you arrive by kayak from the water; but I need to keep going, as I want to meet up with friends later on.

Cape Florida Lighthouse

Cape Florida Lighthouse

I turn the SE corner of my course and head NW again. In the far distance I see the tall buildings in Coconut Grove which is broadly where I need to head. In this section the heat is almost unbearable: The SE breeze is now at my back and also slowing down due to the fallout of the big storm clouds from the West; this creates some very unusual cloud formations and also wide-spread lift which the various birds in the area are leveraging to get up high and continue on their journey without effort.

Near the SW corner of Key Biscayne there are a large number of boats anchoring in shallow water. It is party central, loud music playing, lots of noise and people, mixed with exhaust fumes and beer cans floating in the water. Compared to the quiet journey so far this is the most unusual spot. I often ask myself why people pay much money to get out on the water in a boat, only to then congregate with many others in such a cacophony of loud music and booze. Definitely not my scene. Much to my surprise, there is a small pod of dolphins swimming around here, a welcome contrast to the dozens if not hundreds of boats lining the horizon:

Party Boats and dolphins in foreground

Party Boats and dolphins in foreground

The next leg across Biscayne Bay is the hardest of the trip. The water is choppy, the wind doesn’t help much, I’m tired from by now more than 7 hours on the water, and I’m late for my rendevouz with friends. It is a 10 km straight line distance to get across Biscayne Bay to the Dinner Key Marina. I am getting very tired and also very thirsty. I should stop, drink and call my friends, but somehow I always tell myself that I’ll be there soon and so I continue on. Eventually I get close enough to see various boats and hope to spot Barry and his family who should be anchoring near the West Islands Park. Finally their boat comes into view and I paddle straight to them, fairly exhausted 8h 45min after leaving in the morning.

Relaxing and Refueling on board Plan B, my friend Barry's sailboat

Relaxing and Refueling on board Plan B, my friend Barry’s sailboat

The next two hours are pure bliss: Barry and his family offer up a wide variety of snacks and cold water bottles while we are chatting and relaxing on board his sailboat. We are watching many nice boats go by in and out of the Dinner Key Marina, a boater’s paradise down here. If it was up to me I would not want to leave for many more hours, but I have about another hour of paddling from here to get back to the car, so eventually and reluctantly I hop back into my kayak and set off.

Barry's family and friends waving my off in the evening

Barry’s family and friends waving my off in the evening

Thanks to the still fairly high tide I can round the little Key with just a small sand bank requiring me to walk and push the kayak – making for another scenic photo opportunity.

Around West Islands Park, another hour of paddling to go

Around West Islands Park, another hour of paddling to go

The last hour is slow due to some light headwind and waves, as well as great fatigue in my arms and shoulders. I pass the Mercy hospital where a helicopter lands – probably bringing some accident victim from somewhere in the Miami area. Beyond that is the famous Villa Vizcaya, which Jill and I visited just last year. I take some photo but it’s late and getting dark so they are all blurry. At long last I reach the Rickenbacker Causeway and head straight back to where I had parked the car.

Back at Rickenbacker Causeway, some 12 hours after leaving

Back at Rickenbacker Causeway, some 12 hours after leaving

I am very tired now, but also very content with having closed this loop. Almost 12 hours (8am-8pm) and nearly 9 hours of paddling, with 39 km distance, that’s far and away the longest paddle I have done.

Fatigued, but happy with the day

Fatigued, but happy with the day

Now I have about 1 hour to get to the airport to pick up my son Philip who happens to come back from a 3 week vacation in Austria with his grand parents. I tie down the kayak, eat some snacks and drink several bottles of water and gatorade which I had wisely stashed in the car. The drive home is filled with stories from Philip’s vacation and my long day. I am too tired to unload and wash everything tonight, that will have to wait until tomorrow. Good thing next day is a Sunday, and I can recover at home with sore shoulders and a few blisters, but also some magnificent memories about a day in Biscayne Bay.

 

 

 

 
Leave a comment

Posted by on July 27, 2014 in Recreational

 

Apple’s Health App and HealthKit – Platforms for Next Gen Healthcare?

Originally posted on Business Analytics 3.0:

mobile-applicationsGame on….I think we just witnessed a big next generation leap in Healthcare Data and Analytics.  Apple jumped into the health information business on June 2, 2014, launching both a new health app and a cloud-based health information platform with IOS 8.

The new App, called simply “Health”, will collect a number of body metrics including blood pressure, heart rate, and stats on diet and exercise.  Health will constantly monitor key health metrics (like blood sugar or blood pressure), and if any of them begin to move outside the healthy range, the app can send a notification to the user’s doctor.

The Health app will share all its information with a new cloud platform called “HealthKit.” The new health cloud platform is designed to act as a global repository for all the user’s health information. It will accept data collected by a variety of third-party devices and apps. For instance Nike is now working to…

View original 526 more words

 
Leave a comment

Posted by on June 2, 2014 in Uncategorized

 

Visualizing Voting Preferences for World Values

The other day I listened to a presentation by Melinda Gates prepared for the United Nations to deliver an update about progress towards the Millennium Development Goals (MDG). The eight goals of the MDG had been embraced by the UN back in 2005 for the time target of 2015. So it is reasonable to see whether the world is on track to reach each of these eight goals. To summarize, from the MDG Wikipedia page:

  1. Eradicating extreme poverty and hunger
  2. Achieving universal primary education
  3. Promoting gender equality and empowering women
  4. Reducing child mortality rates
  5. Improving maternal health
  6. Combating HIV/AIDS, malaria, and other diseases
  7. Ensuring environmental sustainability
  8. Developing a global partnership for development

A good listing of reports, statistics and updates can be found on the UN website here.

Sample Vote for 6 of 16 MDG choices

Sample Vote for 6 of 16 MDG choices

At the end of Melinda’s presentation is a link to a UN global survey on the MDG goals after 2015. I took this survey and found the visualization of voting results quite interesting. First, one is asked to select six out of a list of sixteen (6 of 16) goals which one thinks have the highest impact for a future better world. (The survey methodology is described in more detail here.) Here is a sample vote:

A nice touch is that for each of the sixteen goals there is a different color and when you check that goal, one of the sixteen areas on the stylized globe is filled with that color. Personal data such as name is optional, but some demographic information is required, including age, gender, educational level and country. Next, one can look at a summary of all currently tallied votes and compare them interactively to ones own vote (checkmarks on the right).

WorldVoteOverview

It is perhaps not surprising that I voted very similar to others in similar demographic cohorts.

  • Country: I picked five of the Top five goals like all other voters living in the US. I included ‘Political freedoms’ in my top six, which in the US only ranks 11th.
  • Age: I shared five of the Top six goals with people in my age group (world-wide). The one I did not check was ranked 4th (Better job opportunities). When you mouse over one of the goals, the display changes to highlight this goal in all columns:
Interactive Vote Analysis with highlighted goal

Interactive Vote Analysis with highlighted goal

  • Gender: Here I picked four of the Top five goals (did not include the ‘Better job opportunities’).
  • Education: I voted very similar to people with very high HDI (Human Development Index, a visualization of which we covered in a previous post) with five of the Top six.

From the above, it seems somewhat surprising that voters in the US did not ascribe a higher value to ‘Better job opportunities’, given how much economic values and topics like unemployment seem to dominate the media. That said, these votes should be a reflection about which goals are most valuable for making the world a better place – not just your own home country. Worldwide it seems that other, more fundamental goals are judged by voters in the US to be more important than ‘Better Job opportunities’.

Another chart on the results page is showing a heat map of the world countries based on how many votes have been submitted. I thought it was interesting that Ghana had submitted about twice as many votes as all of the US, and Nigeria about 7x as many. The country with most voters at this time is India, but not far ahead of Nigeria.

CountryTotals

A fairly useless dynamic animation in this map is a map pin drop of four people who voted similarly to me. I found this too anecdotal to be of any real interest and downright annoying that I couldn’t turn it off. and just focus on the vote heat map. For example, the total number of votes should be displayed in the Legend. I vaguely remember that it was several hundred thousand from 194 countries prior to starting the survey, but couldn’t get that data to display again without clicking on the Vote Again:

MyWorldVotes

 
Leave a comment

Posted by on September 21, 2013 in Education, Medical, Scientific

 

Personal Analytics with the Suunto Ambit


Suunto Ambit

Half a year ago my wife bought me the Suunto Ambit multi-function sport watch and heart rate monitor. It is a fantastic device, with very precise GPS, lots of add-on functionality and an interesting online portal and community.

There is some configuration and setup involved, such as pairing the Ambit with your heart rate belt and in the case of cycling with a cadence pod. You charge the batteries by plugging it into a USB port, which is also the way how you upload the data form the device to your computer or a website.

While the device itself and its programmability is quite advanced, I want to focus here on the associated online portal called Movescount.com where you can upload and visualize all your data for free – and share it with friends or the community if you’re so inclined. This amount of personal data collecting and analyzing is a fairly recent phenomenon, often referred to as Personal Analytics.

Each recorded session with the Suunto can be uploaded and classified into one of many sports, such as hiking, cycling, basketball, or indoor exercise. Each session is called a move, and with the portal you can collectively visualize all your moves. The current theme at movescount.com has a black background with mostly orange bars and charts. One of the first controls to organize your moves is either a list or a calendar control.

Calendar Control for Moves

Calendar Control for Moves

This already gives you a good overview over the type of sports activities and the distribution over weekday and weekends. A summary display is available in various forms, such as the following simple bar charts.

Summary information about heart rate zones

Summary information about heart rate zones

Another display format summarizes your selected moves, such as all moves in a particular month together with commensurate calorie consumption and breakdown of hours by type of move.

Moves Summary Display

Moves Summary Display

You can now select either a single or multiple moves (or group by the type of move) and display more information about this particular move. Note the x-axis can be set to display either distance or time and one can zoom in on any part of the entire recorded move. One can alos overlay multiple measures in the same chart by selecting more than one factor, although I find this to lead to very busy and confusing charts.

Graph and BarChart details per move

Graph and BarChart details per move

There are many individual measurements available for display, some based on individual sensors (like heart rate or GPS location or temperature or altitude / air pressure), others based on calculations and estimates (such as speed, recovery parameter “R&R”, EPOC or VO2).

MapGraph

Of particular interest to me as a cyclist is the ability to overlay the GPS-track on a Google map. Not only is it a very detailed recording of the route, but it is color-coded based on the currently selected measure. For example, the color-range shows the heart rate in the same colors as the above bar charts. One can clearly see where one is just warming up at the beginning (low heart rate, green color) or where one is riding up “into the red”, i.e. towards the limits of one’s own heart rate. Selecting points along the route displays some information about that particular point of the ride.

One interesting feature would be a time-geo correlation of any portable photo camera when taking pictures along the ride. Based on synchronized time one could then easily geo-code the photos even without any GPS capability within the camera itself.

The Suunto Ambit can do a lot more, including customizing the display mode and storing your configurations in so-called apps. One idea I have for this is to display an estimate of the total calorie consumption for a known route when continuing at the current pace (but I haven’t played with the programming yet). The Ambit seems to be particularly well suited to hiking, mountain biking or skiing due to its altimeter; however I don’t get to leverage that in flat Florida. Only the few bridges over the Intracoastal waterway show up as bumps in the vertical – with the corresponding acceleration of the heart rate on the uphill side.

One of the downsides is the fact that the heart rate sensor worn around the chest does not work in the water. Hence any swimming in the Ocean or the pool can not be measured precisely. (I replace the measurements with estimates.) And sure enough, just recently Suunto announced the new Ambit 2, which overcomes this limitation. Such is the world of new electronic toys, that the half-life of their innovation is getting shorter and shorter.

Bubble Chart of set of rides

Bubble Chart of set of rides

Measures in Bubble Chart

Measures in Bubble Chart

One last chart I wanted to point out is the flexible bubble chart. Shown above is a selection of all my rides in the first half of 2013 (47 rides minus two outliers, very long rides which would have changed the scale and compressed the rest of the chart). This gives a good feel for the distribution and variance of personal rides over a longer period of time – from the quick half hour duration to the more typical rides of a good 2 hours. Note that one can select any of about 30 measures in any of the three drop-down boxes (X-Axis, Y-Axis, Bubble-Size).

One side-effect of measuring and visualizing so many moves is that we find some interesting differences in our respective exercise habits and corresponding energy consumption. While I burn most of my calories on the bicycle, my wife gets more exercise out of indoor circuit exercises and Yoga than I do. For me, after literally decades of recreational cycling, I can raise my heart rate to much higher levels for extended periods of time on the bike compared to indoor circuit exercises. In a way that is not surprising, given the strength and oxygen consumption of the large leg muscles compared to smaller shoulder and arm muscles. But I would not have expected the difference to be so pronounced and could not have quantified it nearly as precisely as without such personal analytics.

It can be expected that the field of healthcare and personal analytics will converge and provide much more personalized data and insight into the specific life of any patient. Medical indicators like heart rate, blood sugar, blood pressure or factors like exercise and diet will become much more quantifiable and individually tracked over time. The hope is that this will also lead to better, more personal and generally more preventive care and medical treatments to any personal condition.

 
Leave a comment

Posted by on June 30, 2013 in Recreational

 

Tags: , ,

Apple, Amazon, Google, Microsoft

Last year we looked at the four companies and compared their business model over two quarters: Apple (hardware), Amazon (retail), Google (advertising), Microsoft (software). It struck me how far the integrated Wolfram Alpha technology has come in the last two years. It combines the symbolic computing capabilities of the Mathematica platform with curated data (for example financial data) and some pretty impressive linguistic analysis capabilities for free-form text input.

For example, in Wolfram Alpha, just type in the following query: “Googe vs. Amazon vs. Apple vs. Microsoft” The results are shown as a series of three screen-shots below:

ComparisonWolframAlpha1

ComparisonWolframAlpha2

ComparisonWolframAlpha3

Not only do you get the various data such as the fundamentals or the analysis of a mean-variance optimal portfolio displayed, but you get all the code needed to programmatically load such data. For example, if you want to get the breakdown of the analyst ratings, the system will expand it as follows:

AnalystRatings

So far we haven’t done any coding or bothered with integrating any data source. This amount of integration and automation is pretty impressive. I am often surprised how few companies are taking advantage of such advanced technology platforms.

 
Leave a comment

Posted by on April 28, 2013 in Financial

 

Tags: , , ,

Visualizing Conversion Rates (Funnels, Bullet Charts)

Most sales processes go through a series of stages, from first contact through successive engagement of the potential client to the close. One can think of these as special cases of attrition-based workflows. These are very typical in online (B2C eCommerce) or tele-marketing (call centers) and companies usually collect a lot of data around the conversion rates at each step. How can one visualize such workflows?

One metaphor for these processes is that of a sales funnel. A lot of leads feed in on one side, and at each stage fewer pass through to the next. It is then straightforward to want to visualize such funnels, such as shown here by Tableau.

Tutorial video explaining how to create a funnel chart using Tableau

Tutorial video explaining how to create a funnel chart using Tableau (Source: Tableau Training Video)

Aside from the somewhat tedious steps involved – custom field calculations, left-right duplication of the chart, etc. – it turns out, however, that funnel charts are not easy to interpret. For example, it is not well suited to answer the following questions:

  • What’s the percentage reduction at each step?
  • Comparing two or more funnels, which has better conversions at each step?
  • What are the absolute numbers in each step?
  • Are the conversion rates above or below targets at each step?

Ash Maurya from Spark59 wrote a very useful article on this topic entitled “Why not the funnel chart?“. In it he looks specifically at comparisons of funnels (current vs. prior time intervals or A|B testing).

Time Series comparison of funnel performance (Source: Ash Maurya's Spark59 Blog)

Time Series comparison of funnel performance (Source: Ash Maurya’s Spark59 Blog)

In a next step he shows that the funnel shape doesn’t add to the readability. Instead simple bar charts can do just as well:

Same information with Bar Charts (Source: Ash Maurya's Spark59 Blog)

Same information with Bar Charts (Source: Ash Maurya’s Spark59 Blog)

For a multi-step funnel, the problem remains that with the first step set to 100%, subsequent steps often have fairly small percentages and thus are hard to read and compare. Suppose you are sending emails to 100,000 users, 30% of which click on a link in the email, of which only 10% (3% of total) proceed to register, of which only 10% (0.3% of total) proceed to subscribe to a service. Bars with 3% or even 0.3% of the original length will be barely visible. One interesting variation is to normalize each step in the funnel such that the new, expected conversion number (or that from the prior period) is scaled back to 100%. In that scenario it is easy to see which steps are performing above or below expectations. (Here big jump in Registrations from Jan to Feb, then small drop in Mar.)

Bar Charts with absolute vs. relative numbers

Bar Charts with absolute vs. relative numbers

Next, Ash Maurya uses the Bullet Chart as introduced by Stephen Few in 2008. The Bullet Chart is a variation of a Bar Chart that uses grey-scale bands to indicate performance bands (such as poor – ok – good) as well as a target to see whether the performance was above or below expectations. The target bar allows to combine two charts into just one, giving a compact representation of the relative performance:

Funnel Chart showing funnel performance (Source: Ash Maurya's Spark59 Blog)

Bullet Chart showing funnel performance (Source: Spark59 Blog)

Various authors have looked at how to create such bullet charts in Excel. For example Peltier Tech has looked at this in this article called “How to make horizontal bullet graphs in Excel“. There is still quite some effort involved in creating such charts, as Excel doesn’t directly support bullet charts. Adding color may make sense, although it quickly leads to overuse of color when used in a dashboard (as Stephen Few points out in his preference for grey scales).

Funnel Graphs in Excel (Source: Peltier's Excel Blog)

Bullet Graphs in Excel (Source: Peltier’s Excel Blog)

Another interesting approach comes from Chandoo with an approximation of a bullet graph in cells (as compared to a separate Excel chart object). In this article “Excel Bullet Graphs” he shows how to use conditional formatting and custom formulae to build bullet graphs in a series of cells which can then be included in a table, one chart in each row of the table.

In-Cell Funnel Graph in Excel (Source: Chandoo's Blog)

In-Cell Bullet Graph in Excel (Source: Chandoo’s Blog)

It is somewhat surprising that modern data visualization tools do not yet widely support bullet charts out of the box.

Measuring how marketing efforts influence conversions can be difficult, especially when your customers interact with multiple marketing channels over time before converting. To that end, Google has introduced multi-channel funnels (MCF) in Google Analytics, as well as released an API to report on MCFs. This enables new sets of graphs, which we may cover in a separate post.

 
Leave a comment

Posted by on March 31, 2013 in Industrial

 

Tags: , , ,

Magic Quadrant Business Intelligence 2013

It’s that time of the year again: Gartner has released its report on Business Intelligence and Analytics platforms. One year ago we looked at how the data in the Magic Quadrant – the two-dimensional space of execution vs. vision – can be used to visualize movement over time. In fact, the article Gartner’s Magic Quadrant for Business Intelligence became the most viewed post on this Blog.

I had also uploaded a Tableau visualization to Tableau Public, where everyone can interact with the trajectory visualization and download the workbook and the underlying data to do further analysis. This year I wanted to not only add the 2013 data, but also provide a more powerful way of analyzing the dynamic changes, such as filtering the data. For example, consider the moves from 2012 to 2013 of some 21 vendors:

Gartner's Magic Quadrant for Business intelligence, changes from 2012 to 2013

Gartner’s Magic Quadrant for Business intelligence, changes from 2012 to 2013

It might be helpful to filter the vendors in this diagram, for example to show just niche players, or just those who improved in both vision and execution scores. To that end, I created a simple Tableau dashboard with four filters: A range of values for the scores of both vision and execution scores, as well as a range of values for the changes in both scores. The underlying data is also displayed for reference, which can then be used to sort companies by ordering along those values.

Here is an example of the dashboard set to display the subset of 15 companies who increased either both or at least one of their vision or execution scores without lowering the other one.

Subset of companies who improved vision and/or execution over the last year.

Subset of companies who improved vision and/or execution over the last year.

That’s more than 70% of platforms, with the increase in vision being more pronounced than that of execution. That’s considerably more than in the previous years (2013: 15; 2012: 6; 2011: 6; 2010: 3; 2009: 9) – making this collective move to the top-right perhaps a theme of this year’s report.

Who changed Quadrants? Who moved in which dimension?

Last year Tibco (Spotfire) and Tableau were the only two platforms changing quadrants, then becoming challengers. This year both of them “turned right” in their trajectory and crossed over into the leaders quadrant due to strong increases in their vision capabilities. (QlikTech had been on a similar trajectory, but already crossed into the leader quadrant in 2012. It also strengthened both execution and vision again this year.)

Another new challenger is LogiXML. Thanks to ease of use, enhancements from customer feedback and a focus on the OEM market its ability to execute increased substantially. From the Gartner report summary on LogiXML:

Ease of use goes hand-in-hand with cost as a key strength for LogiXML, which is reflected by its customers rating it above average in the survey. The company includes interfaces for business users and IT developers to create reports and dashboards. However, its IT-oriented, rapid development environment seems to be most compelling for its customers. The environment features extensive prebuilt elements for creating content with minimal coding, while its components and engine are highly embeddable, making LogiXML a strong choice for OEMs.

A few other niche players almost broke into new quadrants, including Alteryx (which had the biggest overall increase and almost broke into the visionary quadrant), as well as Actuate and Panorama Software.

The latter two stayed the same with regards to execution (as did SAP) – while all three of them moved strongly to the right to improve on the vision score (forming the Top 3 of vision improvement).

Information Builders and Oracle stayed where they were, changing neither their execution nor vision scores.

Microsoft and Pentaho stayed about the same on vision, but increased substantially in their execution scores.  This propelled Microsoft to the top of the heap on the execution score, while it moved Pentaho from near the bottom of the heap to at least a more viable niche player position. Microsoft’s integration of BI capabilities in Excel, SQL Server and SharePoint as well as leveraging of cloud services and attractive price points make it a strong contender especially in the SMB space. Improvements of its ubiquitous Excel platform give it a unique position in the BI market. From the Gartner report:

Nowhere will Microsoft’s packaging strategy likely have a greater impact on the BI market than as a result of its recent and planned enhancements to Excel. Finally, with Office 2013, Excel is no longer the former 1997, 64K row-limited, tab-limited spreadsheet. It finally begins to deliver on Microsoft’s long-awaited strategic road map and vision to make Excel not only the most widely deployed BI tool, but also the most functional business-user-oriented BI capability for reporting, dashboards and visual-based data discovery analysis. Over the next year, Microsoft plans to introduce a number of high-value and competitive enhancements to Excel, including geospatial and 3D analysis, and self-service ETL with search across internal and external data sources.

The report then goes on to praise Microsoft for further enhancements (queries across relational and Hadoop data sources) that contribute to its strong product vision score and “positive movement in overall vision position”. This does not seem consistent with the presented Magic Quadrant, where Microsoft only moved to the top (execution), not to the right (vision). Perhaps another reason for Gartner to publish the underlying coordinate data and finally adopt this line of visualization with trajectories.

Deteriorate2013

Dashboard with filters revealing two platforms deteriorating in both vision and execution

Only two vendors saw their scores deteriorate in both dimensions: MicroStrategy gave up some ranks, but remains in the leader quadrant. The report cites a steep sales decline in 3Q12 and the increased importance of predictive and prescriptive analytics in this years evaluation among the reasons:

MicroStrategy has the lowest usage of predictive analytics of all vendors in the Magic Quadrant. A reason for this behavior might be the user interface that is overfocused on report design conventions and lacks proper data mining workbench capabilities, such as analysis flow design, thus failing to appeal to power users. To address this matter, MicroStrategy should deliver a new high-end user interface for advanced users, or consumerize the analytic capabilities for mainstream usage by embedding them in Visual Insight.

The other vendor moving to the bottom-left is arcplan, which is now at the bottom of the heap in the niche players quadrant.

Who moved to the top-left?

With the dashboard at hand, you can also go back and do similar queries not just for the current year 2013, but any of the five previous years. For example, who has moved to the top-left – improved execution at the expense of reduced vision – over the years?

In 2013 those were Targit, Jaspersoft, Board International. All three of them had a sharp drop in Execution in the previous year 2012. A plausible scenario of what happened is that these companies lost their focus on execution, dropped the scores and in an attempt to turn-around focused on executing well with a smaller set of features (hence lower vision).

In 2012 the only vendor to display a move to the top-left was QlikTech. They had some sales issues the prior year as well, although their trajectory in 2011 was only modestly lower in execution, mostly towards higher vision.

In 2011 Actuate and Information Builders moved to the top-left. Both had trajectories to the bottom-left the prior year (2010), with especially Actuate losing a lot of ground. With the Year slider on the top-left of the dashboard one can then play out the trajectory while the company filters remain, thus showing only the filtered subset and their history. Actuate completed a remarkable turn-around since then and is now positioned back roughly where it was back in 2010.

Dashboard with analysis of top-left moving companies.

Dashboard with analysis of top-left moving companies.

 

(Click on the image above or here to go to the interactive Public Tableau website.)

In 2010 there were five vendor moving to the top-left: Oracle, SAS, QlikTech, Tibco (Spotfire) and Panorama Software. Although in that case none of them did show a decrease in execution the previous year. That focus on execution may simply have been the result of the economic downturn in 2009.

Such exploratory analysis is hard to conceive without proper interactive data visualization. Given the focus of all the vendors it covers in this report, it seems somewhat anachronistic that Gartner in its report does not leverage the capabilities of such interactive visualization itself. In the previous post on Global Risks we have seen how much value that can add to such thorough analysis. (Much of this dashboard should be applicable for risk analysis as well, just that the two-dimensional space changes from platform vision vs. execution to risk likelihood vs. impact!) If Gartner does not want to drop on its own execution and vision scores, they better adopt such visualization. It’s time.

 
1 Comment

Posted by on February 12, 2013 in Industrial

 

Tags: , , , ,

Visualizing Global Risks 2013

Visualizing Global Risks 2013

A year ago we looked at Global Trends 2025, a 2008 report by the National Intelligence Commission. The 120 page document made surprisingly little use of data visualization, given the well-funded and otherwise very detailed report.

By contrast, at the recent World Economic Forum 2013 in Davos, the Risk Response Network published the eighth edition of its annual Global Risks 2013 report. Its focus on national resilience fits well into the “Resilient Dynamism” theme of this year’s WEF Davos. Here is a good 2 min synopsis of the Global Risks 2013 report.

We will look at the abundant use of data visualization in this work, which is published in print as an 80-page .pdf file. The report links back to the companion website, which offers lots of additional materials (such as videos) and a much more interactive experience (such as the Data Explorer). The website is a great example of the benefits of modern layout, with annotations, footnotes, references and figures broken out in a second column next to the main text.

RiskCategories

One of the main ways to understand risks is to quantify it in two dimensions, namely its likelihood and its impact, say on a scale from 1 (min) to 5 (max). Each risk can then be visualized by its position in the square spanned by those two dimensions. Often risk mitigation is prioritized by the product of these two factors. In other words, the further right and/or top a risk, the more important it becomes to prepare for or mitigate it.

This work is based on a comprehensive survey of more than 1000 experts worldwide on a range of 50 risks across 5 broad categories. Each of these categories is assigned a color, which is then used consistently throughout the report. Based on the survey results the report uses some basic visualizations, such as a list of the top 5 risks by likelihood and impact, respectively.

Source for all figures: World Economic Forum (except where noted otherwise)

Source for all figures: World Economic Forum (except where noted otherwise)

When comparing the position of a particular risk in the quadrant with the previous year(s), one can highlight the change. This is similar to what we have done with highlighting position changes in Gartner’s Magic Quadrant on Business Intelligence. Applied to this risk quadrant the report includes a picture like this for each of the five risk categories:

EconomicRisksChange

This vector field shows at a glance how many and which risks have grown by how much. The fact that a majority of the 50 risks show sizable moves to the top right is of course a big concern. Note that the graphic does not show the entire square from 1 through 5, just a sub-section, essentially the top-right quadrant.

On a more methodical note, I am not sure whether surveys are a very reliable instrument in identifying the actual risks, probably more the perception of risks. It is quite possible that some unknown risks – such as the unprecedented terrorist attacks in the US on 9/11 – outweigh the ones covered here. That said, the wisdom of crowds tends to be a good instrument at identifying the perception of known risks.

Note the “Severe income disparity” risk near the top-right, related to the phenomenon of economic inequality we have looked at in various posts on this Blog (Inequality and the World Economy or Underestimating Wealth Inequality).

A tabular form of showing the top 5 risks over the last seven consecutive years is given as well: (Click on chart for full-resolution image)

Top5RisksChanges

This format provides a feel for the dominance of risk categories (frequency of colors, such as impact of blue = economic risks) and for year over year changes (little change 2012 to 2013). The 2011 column on likelihood marks a bit of an outlier with four of five risks being green (= environmental) after four years without any green risk in the Top 5. I suspect that this was the result of the broad global media coverage after the April 2011 earthquake off the coast of Japan, with the resulting tsunami inflicting massive damage and loss of lives as well as the Fukushima nuclear reactor catastrophe. Again, this reinforces my belief that we are looking at perception of risk rather than actual risk.

Another aggregate visualization of the risk landscape comes in the form of a matrix of heat-maps indicating the distribution of survey responses.

SurveyResponseDistribution

The darker the color of the tile, the more often that particular likelihood/impact combination was chosen in the survey. There is a clear positive correlation between likelihood and impact as perceived by the majority of the experts in the survey. From the report:

Still it is interesting to observe how for some risks, particularly technological risks such as critical systems failure, the answers are more distributed than for others – chronic fiscal imbalances are a good example. It appears that there is less agreement among experts over the former and stronger consensus over the latter.

The report includes many more variations on this theme, such as scatterplots of risk perception by year, gender, age, region of residence etc. Another line of analysis concerns the center of gravity, i.e. the degree of systemic connectivity between risks within each category, as well as the movement of those centers year over year.

Another set of interesting visualizations comes from the connections between risks. From the report:

Top5Connections

Top10ConnectedRisks

Finally, the survey asked respondents to choose pairs of risks which they think are strongly interconnected. They were asked to pick a minimum of three and maximum of ten such connections.

Putting together all chosen paired connections from all respondents leads to the network diagram presented in Figure 37 – the Risk Interconnection Map. The diagram is constructed so that more connected risks are closer to the centre, while weakly connected risks are further out. The strength of the line depends on how many people had selected that particular combination.

529 different connections were identified by survey respondents out of the theoretical maximum of 1,225 combinations possible. The top selected combinations are shown in Figure 38.

It is also interesting to see which are the most connected risks (see Figure 39) and where the five centres of gravity are located in the network (see Figure 40).

One such center of gravity graph (for geopolitical risks) is shown here:RiskInterconnections

The Risk Interconnection Map puts it all together:

RiskInterconnectionMap

Such fairly complex graphs are more intuitively understood in an interactive format. This is where the online Data Explorer comes in. It is a very powerful instrument to better understand the risk landscape, risk interconnections, risk rankings and national resilience analysis. There are panels to filter, the graphs respond to mouse-overs with more detail and there are ample details to explain the ideas behind the graphs.

DataExplorer

There are many more aspects to this report, including the appendices with survey results, national resilience rankings, three global risk scenarios, five X-factor risks, etc. For our purposes here suffice it to say that the use of advanced data visualizations together with online exploration of the data set is a welcome evolution of such public reports. A decade ago no amount of money could have bought the kind of interactive report and analysis tools which are now available for free. The clarity of the risk landscape picture that’s emerging is exciting, although the landscape itself is rather concerning.

 
1 Comment

Posted by on January 31, 2013 in Industrial, Socioeconomic

 

Tags: , , , , , , ,

2012 in review

The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.

Here’s an excerpt:

4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 36,000 views in 2012. If each view were a film, this blog would power 8 Film Festivals

Click here to see the complete report.

 
Leave a comment

Posted by on December 31, 2012 in Uncategorized

 

Circos Data Visualization How-to Book

Earlier this year we have looked at a powerful data visualization tool called Circos developed by Martin Krzywinski from the British Columbia Genome Science Center. The previous post looked at an example of how this tool can be used to show complex connectivity pathways in the human neocortex, so-called Connectograms.

Circos Book Cover

The Circos tool can be used interactively on the above website. In that mode you upload jobs via tabular data- and configuration-files and have some limited control over the rendering of the resulting charts. For full expressive power and flexibility, Circos can also be downloaded freely and used on your computer for rendering with extensive customization control over the resulting charts.

I have been asked to review a new book titled “Circos Data Visualization How-to“, published by Packt Publishing here. It’s main goal is to guide through the above download + installation process and get you started with Circos charts and their modification. Here is a brief review of this book.

Although originally developed for visualizing genomic data, Circos has been applied to many other complex data visualization projects, incl. social sciences. One such study was done by Tom Schenk, who analyzed the relationships between college majors and the professions those graduates ended up in. It appears as if this work inspired the author to write this book to help others with using Circos.

I downloaded the book in Kindle format and read it on the Mac due to the color graphics and the much larger screen size. It’s well structured and around 70 pages in printed form. The book focuses first on the download and install part, then has a series of examples from first chart to more complex ones using customization such as colors, ribbons, heat maps or dynamic binding.

Flow Chart for creation of Circos charts

Flow Chart for creation of Circos charts

Circos is essentially a set of Perl modules combined with the GD graphics library.

The first part is on Installing Circos, with a chapter each on Windows 7 and on Linux or Mac OS. Working on MAC I went the latter route. I ended up right in the weeds and it took me about 4 hours to get everything installed and working. The description is derived from a Linux install and is generally somewhat terse. It assumes you have all prerequisite tools installed on your Mac or at least that you are savvy enough to figure out what’s missing and where to get it. I had to dust off some of my Unix skills and go hunting for solutions via Google to a list of install problems:

  • directory permissions (I needed to warp the exact instructions with sudo)
  • installing Xcode tools from Apple for my platform (make was not preinstalled)
  • understanding cause of error messages (Google searches, Google group on Circos)
  • locating and installing the GD graphics library (helpful installing-circos-on-os-x tips by Paulo Nuin)
  • version and location issues (many libraries are in ongoing development; some sources have moved)

Others may find this part a lot easier, but I would say there should be an extra chapter for the Mac with tips and explanations to some of these speed bumps. On the plus side, the Google group seems to be very active and I found frequent and recent answers by Circos author Martin Krzywinski.

The next part of the book is easy to understand. One creates a simple hair-to-eye color relationship diagram. Then configuration files are introduced to customize colors and chart appearance. All required data and configuration files are also contained in the companion download from the Packt Publishing book page.

Chart of relationship between hair and eye colors

Chart of relationship between hair and eye colors

The last part of the book goes into more advanced topics such as customizing labels, links and ribbons, formatting links with rules, reducing links through bundling, and adding data tracks as heat maps or histograms. This is the meat for those who intend to use Circos in more advanced ways. I did not spend a lot of time here, but found the examples to be useful.

Contributions by State and Political party during 2012 U.S. Presidential Elections

Contributions by State and Political party during 2012 U.S. Presidential Elections

This section ends abruptly. One gets the feel that there are other subtleties that could be explored and explained. A summary or outlook chapter would have been nice to wrap up the book and give perspective. For example, I would have liked to hear from the author how much time he spent with various features during the college major to professions project.

In summary: This book will get you going with Circos on your own machine. Installing can be a challenge on Mac, depending on how familiar you are with Unix and the open source tool stack. The examples for your first Circos charts are easy to follow and explain data and configuration files. The more advanced features are briefly touched upon, but require more experimentation and time to understand and appreciate.
Circos author Martin Krzywinski writes on his website: “To get your feet wet and hands dirty, download Circos and a read the tutorials, or dive into a full course on Circos.” The How-to book by Tom Schenk helps with this process, but you still need to come prepared. If you are a Unix power user this should feel familiar. If you are a Mac user who rarely ever opens a Terminal then you might be better off just using Circos via the tableviewer web interface.
Lastly, I would recommend buying the electronic version of this book, as you can cut & paste the code, leverage the companion code and documents. A printed version of this book would be of very limited use.

 
1 Comment

Posted by on December 6, 2012 in Education, Scientific

 

Tags: , , ,

 
Follow

Get every new post delivered to your Inbox.

Join 102 other followers

%d bloggers like this: