Data Science and Impact Research

---

Data Science is central to our work in two ways:

Focused Research: We research and write about data science and the effect it can have on human development. Advances in the analysis and application of data are driving innovation across nearly every industry

Data Science is often poorly understood and badly misrepresented in the mainstream media. Machine learning, for example, remains a black box to nearly everybody who doesn’t actually know how to code. It is often the case that those who understand it don’t write about it - and those who write about it don’t have experience in the field.

We are both practitioners in the field and journalists with the skill set to deliver clear and lucid insights into this complex domain.
​

Applied Methodologies: We apply data science methodologies to uncover hidden insights, ensure our work is statistically rigorous, and to illuminate our research through data visualizations.

Our team is familiar with a wide range of techniques, including applied machine learning, natural language processing, statistical methodologies, and data visualization. Our experience in the field ranges from hands-on application, to the management of technical teams, to strategy and leadership in the boardroom.​​

Data science lab

Data Science Research Techniques

---

Data Science and machine learning allow researchers to extract hidden insights.

​The descriptions below are intended to provide a helpful overview of techniques we use to apply machine learning to the field of impact research. Out of necessity, they are simplifications of very complex subjects.

Data wrangling and cleaning
Over 90% of a typical data scientist's time is spent wrangling and cleaning. Data wrangling is the process of transforming raw data into a usable format for analysis. Data cleaning is the process of addressing errors and inconsistencies in the initial data sets. This is important because real world data is often messy and incomplete - a byproduct of real world interactions. Data scientists must transform the data they have into the data they need.

Exploratory data analysis
EDA is how a data scientist gets to grips with a new data set in order to make it useful. Data scientists must develop an intuitive sense for how to use statistical methodologies and data visualization techniques. This is important in order to explore a new data set and develop a strategy from how to extract insights from it. 

Feature selection and engineering
Feature selection is how a data scientist chooses and refines which data they’re going to work with. In more technical terms, it is the process of applying rigorous scientific methodology to the curation, selection, and transformation of data which will be used as inputs in machine learning algorithms.

Model design and algorithm selection
Machine learning is a set of tools which allows computers to generate predictions about the world around us and find correlations and complex insights across data from nearly any domain.
A data scientist will often test multiple algorithms and choose the one best fitted to the data, depending on the requirements of the project.

Error metric selection
Data scientists much choose an appropriate measurement to evaluate the performance of each model. They have a wide range of statistical tests at their disposal, but their selection must be non-arbitrary, defensible, and made before they design the model. 

Analysis of output
Fundamentally, the ‘science’ in data science comes from a rigorous application of the scientific method. The final step after every experiment is to analyze the results and decide on the next steps.