The Fundamentals of Data Science


Two of the biggest buzzwords in our industry is “big data” and “data science”. Big Data seems to have a lot of interest right now, but Data Science is fast becoming a very hot topic.

I think there’s room to really define the science of data science - what are those fundamentals that are needed to make data science truly a science we can build upon?

Below are my thoughts for an outline for such a set of fundamentals:

Fundamentals of Data Science

- What is Data?
- The Goal of Data Science
- The Scientific Method

Probability and Statistics
- The Two Characteristics of Data
- Examples of Statistical Data
- Introduction to Probability
- Probability Distributions
- Connection with Statistical Distributions
- Statistical Properties (Mean, Mode, Median, Moments, Standard Deviation, etc.)
- Common Probability Distributions (Discrete, Binomial, Normal)
- Other Probability Distributions (Chi-Square, Poisson)
- Joint and Conditional Probabilities
- Bayes’ Rules
- Bayesian Inference

Decision Theory
- Hypothesis Testing
- Binary Hypothesis Test
- Likelihood Ratio and Log Likelihood Ratio
- Bayes Risk
- Neyman-Pearson Criterion
- Receiver Operating Characteristic (ROC) Curve
- M-ary Hypothesis Test
- Optimal Decision Making

Estimation Theory
- Estimation as Extension of M-ary Hypothesis Test
- Unbiased Estimation
- Minimum Mean Square Error (MMSE)
- Maximum Likelihood Estimation (MLE)
- Maximum A Posteriori Estimation (MAP)
- Kalman Filter

Coordinate Systems
- Introduction to Coordinate Systems
- Euclidian Spaces
- Orthogonal Coordinate Systems
- Properties of Orthogonal Coordinate Systems (angle, dot product, coordinate transformations,
- Cartesian Coordinate System
- Polar Coordinate System
- Cylindrical Coordinate System
- Spherical Coordinate System
- Transformations Between Coordinate Systems

Linear Transformations
- Introduction to Linear Transformations
- Properties of Linear Transformations
- Matrix Multiplication
- Fourier Transform
- Properties of Fourier Transforms (time-frequency relationship, shift invariance, spectral
properties, Perseval’s Theorem, Convolution Theorem, etc.)
- Discrete and Continuous Fourier Transforms
- Uncertainty Principle and Aliasing
- Wavelet and Other Transforms

Effects of Computation on Data
- Mathematical Representation of Computation
- Reversible Computations (Bijective Mapping)
- Irreversible Computations
- Impulse Response Functions
- Transformation of Probability Distributions (due to addition, subtraction, multiplication,
division, arbitrary computations, etc.)
- Impacts on Decision Making

Prototype Coding / Programming
- Introduction to Programming
- Data Types, Variables, and Functions
- Data Structures (Arrays, etc.)
- Loops, Comparisons, If-Then-Else
- Functions
- Scripting Languages vs. Compilable Langugages
- R
- Python
- C++

Graph Theory
- Introduction to Graph Theory
- Undirected Graphs
- Directed Graphs
- Various Graph Data Structures
- Route and Network Problems

- Introduction to Algorithms
- Recursive Algorithms
- Serial, Parallel, and Distributed Algorithms
- Exhaustive Search
- Divide-and-Conquer (Binary Search)
- Gradient Search
- Sorting Algorithms
- Linear Programming
- Greedy Algorithms
- Heuristic Algorithms
- Randomized Algorithms
- Shortest Path Algorithms for Graphs

Machine Learning
- Introduction to Machine Learning
- Linear Classifiers (Logistic Regression, Naive Bayes Classifier, Support Vector Machines)
- Decision Trees (Random Forests)
- Bayesian Networks
- Hidden Markov Models
- Expectation-Maximization
- Artificial Neural Networks and Deep Learning
- Vector Quantization
- K-Means Clustering

Question:  Do you have any thoughts on the fundamentals of data science? You can leave a comment below.

A Data Science Lesson from Richard Feynman

Richard Feynman

Richard Feynman

Richard Feynman is one of the greatest scientific minds, and what I love about him, aside from his brilliance, is his perspective on why we perform science.   I’ve been reading the compilation of short works of Feynman titled The Pleasure of Finding Things Out, and I recently came across a section that really hit home with me.

In the world of data science, much is made about the algorithms used to work with data, such as random forests or k-mean clustering.  However, I believe there is a missing component – one that deals the fundamentals underlying data science, and that is the real science of data science.

10 Things To Know When Hiring Data Scientists

I’ve been performing data science before there was a field called “data science“, so I’ve had the opportunity to work with and hire a lot of great people.  But if you’re trying to hire a data scientist, how do you know what to look for, and what should you consider in the interview process?

Data Science Word Cloud

I’ve been doing what is now called “data science” since the early 1990s and have helped to hire numerous scientists and engineers over the years.  The teams I’ve had the opportunity to work with are some of the best in the world, tackling some of the most challenging problems facing our country.  These folks are also some of the smartest people I’ve ever had the opportunity to work with.

That said, not everyone is a good fit, and the discipline of data science requires important key elements.  Hiring someone into your team is incredibly important to your business, especially if you’re a small startup or building a critical internal data science team; mistakes can be expensive in both time and money.  This can be even more intimidating if you don’t have the background or experience in hiring scientists, especially someone responsible for this new discipline of working with data.

The Best Way To Learn New Things

Science and business seem like two very different disciplines, but is the best approach to learning any different in these two fields?  These areas of life seem so unique, and the people in them can be quite varying (one with the nerdy pocket protector and the other dressed in the well-tailored suit).  However, both science and business require learning, and the best approach to learning in each is really the same.

The best approach to learning is generally through failure.  For example, Thomas Edison failed an astounding number of times before he invented a working lightbulb, and there are likely thousand of stories about how successes came as a result of many tries and many failures.

In many ways, this is really an application of the scientific method.  I’ve written a number of posts about Stephen Wolfram (such as using Wolfram|Alpha to look at your own social network, his views on big data, computing a theory of everything, and how he created his company).  In the effort to learn even more about how the world works, Wolfram has pushed scientific discovery to the next level, which he’s done with his book A New Kind of Science (NKS for short).

Beating Cancer and Favoritism with Data

Fortune-Startup RisingI read a couple of items in this month’s Fortune magazine that I thought it was worth passing along.

The first was a small article by Brian Dumaine about the work being done at Applied Proteomics to identify cancer before it develops.  At Applied Proteomics, they use mass spectroscopy to capture and catalog 360,000 different pieces of protein found in blood plasma, and then let supercomputers crunch on the data to identify anomalies associated with cancer.  The company has raised $57 million in venture capital and is backed by Microsoft co-founder Paul Allen.  You can read the first bit of the article here.

The second is from the Word Check callout, showing how access to information is making the word a better place:

wasa: Pronounced [wah-SUH]

(noun) Arabic slang:  A display of partiality toward a favored person or group without regard for their qualifications.  A system that drives much of life in the Middle East — from getting into a good school to landing a good job.

But on the Internet, there is no wasa.

- Adapted from Startup Rising: The Entrepreneurial Revolution Remaking the Middle East by Christopher M. Schroeder

8 Lessons from Nerd Culture

SupermanI found this set of business wisdoms in the August 2013 issue of Entrepreneur magazine.  While not perfect mantras by which to guide a business, I thought there were pretty fun.


Chris Hardwick didn’t rely on just his nerdy instincts in founding his company; he also took inspiration from his heroes.  Super-power your business with these lessons from some epic nerd properties.

How Wolfram|Alpha Can Help You Discover Your Own Social Network

Ever wonder what your own personal network looks like?  You are likely connected to many different groups (family, friends, community, work), but do you know how they are connected?  Or are they connected at all?  Are you the glue that connects these various groups?

Word Cloud

This is a great age we’re living in, and I’m glad to be involved with developing lots of really advanced technologies.  One of the technology areas that I’m really fascinated with has been pushed forward by Stephen Wolfram.  He created the industry standard computing environment Mathematica, which now serves as the engine behind his company’s newest creation, Wolfram|Alpha.  (I’ve written a few posts on Wolfram|Alpha in the past, and you can read them here and here).

How to Make Your Own Custom URL Shortener

This is a technical post about what I’ve discovered in creating my own custom URL shortener.  Hopefully, you can learn to do the same things I did, and my experience will save you some headaches if it’s something you’re interesting in trying.


On my website, I focus a lot about decisions and discovery.  I love finding out how the world works and then applying what I’ve learned to make better decisions, and I also try to share what I can along the way.  I hope that it helps others.

How to Make Better Decisions: Recognize Uncertainty

It’s a complex world, and we are constantly making decisions.  Just imagine the number of decisions we make about breakfast:  How big a breakfast should I have?  Should I have coffee?  If so, how much?  Should I have toast?  Should I use butter?  Should I have one piece or two?  Should I cut the toast?  If so, should they be cut into rectangles or triangles?  Should I keep the crust? Should I have juice?  Should it be apple juice or orange juice?  How about milk?  I haven’t even gotten to the pancakes, waffles, syrup, sausage, cereal, bacon… (mmm, bacon…)


And these aren’t the really important ones!  How do we know we’re making good decisions, and can we make better ones?

5 Reasons to Collect Data on Yourself

You might think that it’s a bit odd, treating yourself like a science experiement.  However, the best way to achieve your goals may be to do just that – be committed to collecting data on yourself.

Chalk Chart

In science, we’re always collecting data and analyzing it to find out more about the world.  However, collecting data isn’t only for people with pocket protectors (although we don’t all wear those!).  It is something that any of us can use to help us achieve any goal we set for ourselves.