Here are some interesting data science nuggets that I thought were interesting for a mid-January day…

The first comes from TechMASH about data science being the next big thing.  The primary nugget of note is that the supply of employees with the needed skills as data scientists – those people who really understand how to pull relevant information out of data reliably – is going to have a tough time meeting demand.  Here’s an interesting infographic on the current disconnects – for example, while 37% of “business intelligence” professional studied business in school, 42% of today’s “data scientists” studied computer science, engineering, and natural sciences.  This highlights the increasing demand for students that have solid mathematics backgrounds – it’s becoming more about knowing how you pull information from data, regardless of application.

Don’t get me wrong – to be effective applying data science, you need two things:  a subject matter expert that understands what makes sense and what doesn’t, and someone who really understands data to pull out the information.  Sometimes that can reside within one person, but it’s rare and takes many years of training to acquire the necessary excellence in both fields.   And as the demands for data analysis grow, these two areas will likely form into distinct disciplines with interesting partnership opportunities being created.

The definition of data science is still being defined, but I’m convinced it will have huge impact in the next five years.  And while the science aspects of data are starting to be defined, the engineering aspects of data and analytics are truly in their infancy…

On the same thread, here’s a Forbes article by Tom Groenfeldt on the need for data scientists, or Excel jockeys, or whatever they will be called in the future.  For some companies, the move to “data science” is quite apparent, but for others, the current assemblance of business professionals that have figured out the ins-and-outs of Excel spreadsheets work quite well.  This is likely a snapshot of where things are today, but I do believe that as the questions we ask of the data get more complicated, we will clearly see the need for a more rigorous science-based discipline to data wrangling…

The last tidbit is from the Wall Street Journal about the healthcare field being the next big area for Big Data.  I do think that healthcare is ripe for leveraging data, and I’ve written other posts on the subject.  One former Chief Medical Officer that I spoke with mentioned that one of the big problems is just getting the data useable in the first place.  He said that, as of today, 85% of all medical records are still in paper form.  The figure seems a bit high to me, but I don’t really know how many patient records in various individual doctor’s offices are still sitting in folders on shelves. 

There has been a big push lately, spurred by financial support from the U.S government, for upgrading to electronic health records (EHR).   This will help to solve the data collection problem – if you can’t get data into an electronic format, you can’t utilize information technologies to pull information out of the data.

I ran across this article from the Independent today about the impacts of data algorithms, the ethics of data mining, and the future of our lives in an automated, data-crunching world.  Below is a quote from the article by Jaron Lanier, musician, computer scientist and author of the bestseller You Are Not a Gadget.

Algorithms themselves are a form of creativity. The problem is the illusion that they’re free-standing. If you start to think that information isn’t just a mask behind which people are hiding, if you forget that, you’ll pay a price for that way of thinking. It will cause you to be less creative.

If you show me an algorithm that dehumanises, impoverishes, manipulates or spies upon people,” he continues, “that same core maths can be applied differently. In every case. Take Facebook’s new Timeline feature [a diary-style way of displaying personal information]. It’s an idea that has been proposed since the 1980s [by Lanier himself]. But there are two problems with it. One, it’s owned by Facebook; what happens if Facebook goes bankrupt? Your life disappears – that’s weird. And two, it becomes fodder for advertisers to manipulate you. That’s creepy. But its underlying algorithms, if packaged in a different way, could be wonderful because they address a human cognitive need.

I think this is a really great read for anyone who’s interested in data, algorithms, and their impact on society – there’s a lot of really good stuff to take in.  You can read the entire article here…

This is the very question asked by Colin Hill, CEO and co-founder of GNS Healthcare, a healthcare analytics company.  Hill hopes to make the case that healthcare can benefit from what a recent McKinsey report calls “the next frontier for innovation, competition and productivity.”  

I think Hill is onto something, especially with this insight:

What will healthcare look like in the year 2020?  One thing is certain: we can’t afford its current trajectory.  Left unchecked, our $2.6 trillion in annual spending will grow to $4.6 trillion by 2020, one-fifth of GDP.  With almost 80 million Baby Boomers approaching retirement, economists forecast these trends will likely bankrupt Medicare and Medicaid in the near future.  And while healthcare reform ignites a number of important changes, alone it does not resolve our issues.  It’s critical we fix our system now.

Something’s got to give, and better decisions from better data can yield significant healthcare savings if done right.  Saving lives and reducing costs dramatically in healthcare would qualify as one of those hard problems where disciplined approaches can yield significant results.  Here is Hill’s post on Forbes…

I ran across this interview by Fast Company with LinkedIn co-founder about his new book The Start-Up of You and the need for companies to have a data strategy, or risk losing “potentially a lot” in the future.  Here’s that brief bit from the Hoffman interview:

What do companies miss out on if they don’t have a data strategy?

Potentially a lot. If you say the way our products and services are constituted, how we determine our strategy and maintain a competitive edge against other folks–if data is a very strong element of each of these, and you’re not doing anything, it’s like trying to run a business without business intelligence. I’m not sure I have a broad enough view that I would say every company needs to have a data strategy. But I would say many companies do. I certainly think that any company that is over 20 people needs to have a technology strategy, and data is essential to where technology is going.

LinkedIn has already been on record as not worrying about Facebook taking over their business.  According to Hoffman, “People with advanced degrees are three times more likely to use LinkedIn.”

You can read the Fast Company interview here…

Are banks predicting divorces?  Well, if there’s data to help them predict such things, they may very well use it to optimize their business.

Forbes has a couple of posts that peek into businesses use of “big data”.  The first article talks about the race to build new analytics to solve challenges of large volumes of data.  Here’s a snippet from Tom Groenfeldt‘s post, quoting Scott Gnau, head of research and development at Teradata:

Thought leaders in a number of industries are starting to leverage the additional analytic content from big data and combine it with what they have in large volume data stores as well. It is interesting to understand social media and consumer sentiment, but when that information is analyzed in combination with traditional consumer data it provides new, rich intelligence helping companies to identify trends and react to immediate business conditions.

According to another Forbes article, there are a number of studies that show that companies that characterize themselves as ”data driven” as the best corporate performers.  Now, when we’re talking “data driven”, we mean in how the company operates, not necessarily in what it produces as technology.  Top performing companies are determined to use the data that they have (especially about themselves) to improve what they do and how they do it. 

Also, banks are on the lookout for changes that could affect how they do business with their customers, and of course, their bottom line:

Banks, for example, worry about their customers divorcing, because divorce causes a change in credit-worthiness. No problem. They can now see a divorce coming before the couple does. All from the data.

As part of the “Computer Science or Data Science” panel at Techonomy 2011 in Tucson, AZ this week, the panel explored how data science has taken its place next to computer science as a fundamental element of information technology.  New technologies are coming out seemingly every day, not only to handle big data, but to understand how to extract relevant information from the ocean of data we’re swimming in.

A company in Silicon Valley, ai-one, announced today that they have “a breakthrough method to graphically represent knowledge enables software developers to easily build intelligent agents such as Apple’s SIRI and IBM Watson”.  The technology, ai-Fingerprint, is geared toward natural language programming, allowing developers to create new technologies that use natural language as input data.  

Apple’s Siri and IBM’s Watson are definitely heading in the right direction for this type of technology.  I just bought an iPhone 4S and I’ve tested Siri out a number of times.  While Siri doesn’t get everything right (it keeps thinking my name is “Nick” when I say “Mic”), it does get more right than I expected.  I was able to send texts and e-mails to people without keystrokes, and I took some notes using the voice feature, getting nearly every word correct.  Pretty amazing stuff!…

Watson is the supercomputer that beat two longtime Jeopardy! champions, and it uses a technology approach that looks for the best answer for the questions being asked (or in this case, the best question for the answer being presented – it is Jeopardy! after all…).  These are definitely the models that should be emulated; although, ai-one’s announcement is a press release so before we see the results, let’s chalk this up at the moment as good marketing…

Edd Dumbill is the general manager for the Strata Conference, recently wrote a nice post on Google+ titled “Why Do We Need Data Science?”

Here is a really good insight from Dumbill and how data science applies to business:

Why is the scientific method applicable to business and data?

Every company’s business is complex in itself, and they operate in a complex world. The financial, economic and societal structures we live and do business in are complex. Because of this complexity and interactions, businesses can be viewed in the same light as organic, biological systems. They are complex entities within a complex system.

This is where science comes into play. Even assuming you could come up with a top-down mathematical model of your business, there’s too much interaction and randomness with complex systems for your model to be practical. Thus, the exploratory approach of science becomes useful to a business.

Your world and business is a giant laboratory, and ever more so as the world becomes more networked. By employing data scientists you can discover better how your business works, how it can be improved, and find new things you can do that you didn’t know of before. To do this, you must connect up three kinds of people: the business folk, the data scientists and the data engineers.

I do like Dumbill’s take here and there is absolute merit with applying the scientific method to business activities.   Peter Wang of Streamitive commented on Dumbill’s post as well, and has some interesting points…

Ultimately, data doesn’t mean anything without trying to answer questions.  To get actionable information, you need data and you need to be asking the right questions. That’s why the scientific method is so important – it’s all about posing a hypothesis or asking a question, and then squeezing the right information out of the data in order to answer it.

A couple of interesting notes today….  On the PRNewswire today, Kontagent, a leading enterprise user analytics company, today announced that it has closed $12 million in Series B financing with a consortium of investors including Battery Ventures, Altos Ventures, and Maverick Capital.  Kontagent focuses on social and mobile web applications with their kSuite product, which combines a proprietary database with customized analytics and real-time monitoring to help customers identify and react to usage patterns in real-time.  This continues the pattern of heavy entry level funding into analytics and big data startups – data science applications are becoming the next big technology boom…

There’s another interesting article snippet at Bloomberg Businessweek about how the oncoming avalanche of data could change the nature of astronomy and physics.  According to the article, Johns Hopkins will be building a 100 gigabit-per-second network to shuttle data from the campus to other large computing centers at national labs and even to Google.  Here’s what Dr. Alex Szalay, Alumni Centennial Professor at Johns Hopkins and head of the network project, thinks about what this could mean for the future of science itself:

In his mind, the new way of using massive processing power to filter through petabytes of data is an entirely new type of computing that will lead to advances in astronomy and physics, much as how the microscope’s creation in the 17th century led to advances in biology and chemistry. In that light, the creation of a 100 gigabit-per-second research network at Johns Hopkins becomes not just a fast network but also an essential tool for research and discovery, a basic component of the 21st century microscope.

You can read about the Kontagent financing deal here, and Dr. Szalay’s effort to build big data networks here…

I ran across this post on Analyst First by Stephen Samild, where he describes the world of Business Analytics as a prime area for the Lean Startup model.

I am a big fan of Eric Ries’ book The Lean Startup, where he advocates treating every new entrepreneurial venture, whether inside an existing company or as its own startup company, as a startup.  And further, since this is a startup venture, the uncertainty about whether this venture will succeed or fail is very high.

So, rather than put together detailed plans about building the product that the business will be based upon, a startup venture should be building the “minimum viable product” and getting feedback from customers quickly to see whether you’re on the right track.  The faster you get feedback, the better you’ll be able to build a business that is sustainable and meets the needs of your customers.

Effectively, Ries argues that you should treat the startup venture, every aspect of it, as a series of scientific experiments designed to inform you whether you are building a sustainable business consistent with your company’s vision.  It’s basically applying the scientific method to your business.

For one, I say, “Absolutely dead on!”  Most business activities, whether marketing or sales or even less-than-disciplined engineering, are performed via rules of thumb (“here’s what’s worked before…”) - there is no true “validated learning”, as Ries put it.   Generally, many businesses and engineering teams operate with the approach of “we made a number of changes last month, and our customers seem to like them, and our overall numbers are higher this month, so we must be on the right track”.  This might make a company feel good, but it gets them no closer to understanding why they might be succeeding, and what to do if things turn south.

And what is worse, the internal workings of the business may be driven by managers more motivated by preserving the current business enterprise than creating a new one.  This puts entrepreneurial ventures at risk from getting started in the first place, or at least started with the greatest possible chance for success.

And I like the way that Samild describes it in his post:

In the twenty-first century we can build almost anything that can be imagined. The challenge is not to build more stuff. It’s to build the right stuff. Most startups fail, says Ries, because they make the wrong things. The key activity of a startup should therefore be learning, not building. What creates value for a startup is it determining whether or not it’s on the path to a sustainable business.

If you’re interested in the Lean Startup approach to business (which, again, I highly recommend), you can find out more at Eric Ries’ website here, and you can buy his book The Lean Startup here…  Also, you can read more of Stephen Samild’s post here…