Top 5 Data Science GitHub Repositories and Reddit Discussions (February 2019)

Introduction

GitHub has become an essential platform for data scientists to share, collaborate on, and showcase their projects. With top companies like Google, Facebook, and OpenAI regularly open-sourcing state-of-the-art models and tools, GitHub provides an invaluable pulse on the bleeding edge of data science and AI.

As an AI and ML expert, I can confidently say that regularly exploring what‘s trending on GitHub is one of the best ways to stay on top of the lightning-fast pace of progress in this field. You‘ll find implementations of new approaches from the latest research papers, extensive codebases and datasets from tech giants, and innovative projects from the vibrant open-source community.

In this post, I‘ll be highlighting my top picks from GitHub in February 2019 that every data scientist should check out. I‘ll dive into the technical details and potential applications of each repository, sharing my own analysis and perspective. I‘ll also summarize some thought-provoking discussions from the Reddit data science community on key trends and pain points.

1. StyleGAN – Generating Hyper-Realistic Human Faces

Developed by NVIDIA, StyleGAN is a groundbreaking generative model that can create ultra-realistic fake faces nearly indistinguishable from real photos. It‘s an evolution of the popular Progressive Growing of GANs (PGGAN) architecture.

The key innovation is the introduction of an "intermediate latent space" that allows for much finer control over the style of the generated images at different scales. This leads to amazingly coherent and high-quality results with 1024×1024 resolution.

Some of the potential use cases include:

• Creating synthetic datasets for facial recognition systems
• Generating characters for video games and movies
• Visualizing different characteristics (age, gender, etc.) for faces

The authors also proposed two new evaluation metrics, Perceptual Path Length and Linear Separability, to quantify the quality of the generated images. They used these to compare StyleGAN to PGGAN on several datasets:

Dataset	PGGAN PPL	StyleGAN PPL	PGGAN LS	StyleGAN LS
CelebA-HQ	2352.2	146.7	3.79	16.6
LSUN Bedroom	2509.5	160.6	1.95	6.37
LSUN Car	3443.9	338.1	3.75	16.2
LSUN Cat	2857.1	73.3	2.22	34.8

PPL = Perceptual Path Length, LS = Linear Separability (higher is better for both)

As we can see, StyleGAN significantly outperforms PGGAN on both metrics across all datasets. However, there is still much room for improvement, especially in terms of reducing "unnatural" artifacts and increasing the diversity/fidelity of fine details.

Generating realistic human faces also raises important ethical considerations around consent, privacy, and misuse. We‘ll need to have ongoing discussions as a community to establish responsible practices as this technology becomes more accessible and mainstream.

2. GPT-2 – Pushing the Boundaries of Language Modeling

In February, OpenAI sent shockwaves through the NLP world with the release of GPT-2, a massive unsupervised language model trained on 40GB of web text. By leveraging advances in transformer architectures and scaling up to 1.5 billion parameters, GPT-2 achieves state-of-the-art performance on several language tasks.

Some remarkable examples of what GPT-2 can do:

• Generate long, coherent articles, stories, and even poetry
• Summarize and translate text
• Answer open-ended questions
• Engage in dialogue

The samples are so convincing that OpenAI decided to only release a smaller version of the model due to concerns about malicious use (e.g. generating fake news, impersonating people). This sparked a lot of discussion around the responsible development of ever-more powerful language models.

GPT-2 was trained on a simple objective (predicting the next word given previous words) but can be adapted to various downstream tasks through fine-tuning or prompt engineering. The key hyperparameters are:

• Model size: 124M, 355M, 774M, 1558M parameters
• Batch size: 512
• Training steps: 1M
• Learning rate: 0.0001
• Optimizer: Adam with weight decay of 0.01
• Dropout: 0.1
• Warmup steps: 2000
• Max sequence length: 1024

While the smaller model is a good starting point, training the full GPT-2 requires significant compute resources (256 TPU v3 cores for several weeks, as per the paper). This highlights the advantage that well-resourced labs have in pushing the boundaries of language modeling.

Some potential applications of GPT-2 include:

• AI writing assistants
• Chatbots and virtual agents
• Content creation and summarization
• Enhancing search engines
• Creative writing support

It will be interesting to see how OpenAI responsibly shares GPT-2 and future language models. Other researchers have already replicated the model architecture, so it may only be a matter of time before even larger models are widely available.

3. DeOldify – Colorizing and Restoring Old Images with Deep Learning

Have you ever wanted to bring old black-and-white photos back to life with color? That‘s exactly what DeOldify does using a novel GAN-based architecture. Developed by Jason Antic, DeOldify combines the task of colorization with the goal of restoring old images that have undergone degradation.

Some key features of DeOldify:

• Self-attention to capture long-range dependencies in the image
• Progressive growing (inspired by StyleGAN) to stabilize training
• Perceptual loss based on feature maps of a pretrained classifier
• Multi-scale discriminators at different resolutions
• Used ResNet34 and UNet architectures for generator and critic

The training data consists of millions of color/B&W image pairs, sourced from ILSVRC2012, BSD300, and unsplash.com. The model can colorize images in a wide range of styles, from photorealistic to artistic, by tuning the rendering resolution and backpropagation steps.

In addition to photos, DeOldify has shown impressive results on old film footage, able to restore both the color and sharpness. Some exciting applications include:

• Bringing historical photos and videos to life
• Enhancing scanned documents and archives
• Aiding film/video restoration efforts
• Providing inspiration for artists and designers

Of course, the colorization is not always perfect and may sometimes yield unnatural results. It‘s still an active area of research to improve the color accuracy, spatial consistency, and generalization to unseen data. But DeOldify demonstrates the remarkable progress in this task using modern deep learning techniques.

4. MindsDB – Automated Machine Learning in SQL Databases

MindsDB is an exciting open-source project that aims to democratize machine learning for database users. It provides an easy way to automatically build and deploy ML models straight from the database using only SQL commands.

The key idea is to enable database users to make predictions from their data without having to go through the traditional process of exporting data to a separate environment, training models, and deploying them back into the database. MindsDB essentially brings the machine learning to the data.

Some highlights of MindsDB‘s architecture and capabilities:

• Connects to any SQL database (MySQL, PostgreSQL, MariaDB, etc.)
• Supports supervised learning tasks like classification and regression
• Automatically handles data preprocessing, model selection, and hyperparameter tuning
• Utilizes LightGBM, CatBoost, and TensorFlow behind the scenes
• Exposes a REST API and database integrations for easy deployment
• Provides explainable predictions with feature importances

Here‘s an example of what a typical workflow looks like in SQL:

-- Create a new predictor
CREATE PREDICTOR mindsdb.sales_model
FROM mydb.sales_data
PREDICT customer_ltv
USING 
    customer_id, 
    total_purchases, 
    signup_date,
    acquisition_channel
;

-- Make a prediction
SELECT customer_id, sales_model.customer_ltv
FROM mydb.new_customers
JOIN mindsdb.sales_model
WHERE signup_date > ‘2019-01-01‘
;

This drastically lowers the barrier to machine learning for database professionals and enables more widespread adoption in enterprises. Some promising use cases include:

• Demand forecasting
• Fraud detection
• Customer churn prediction
• Dynamic pricing
• Predictive maintenance

MindsDB is still a relatively young project and has limitations in the types of data and models it supports. But the automated, SQL-centric approach is a powerful paradigm that could shape the future of applied machine learning. It will be exciting to see how the project evolves and makes ML more accessible to the masses.

5. Uber Ludwig – Declarative Deep Learning Framework

Ludwig is Uber‘s open-source declarative deep learning framework that allows users to train and deploy models without writing any code. It aims to abstract away the complexity of traditional deep learning frameworks and make it easy to build models for a wide range of tasks.

The key idea behind Ludwig is to separate the model definition from the implementation details. Users only need to provide a YAML configuration file specifying the input features, output target, and model architecture. Ludwig then takes care of the rest, from data preprocessing to model training to evaluation.

Some of the key features of Ludwig:

• Supports text, image, audio, and tabular data
• Provides a wide range of model architectures (CNNs, RNNs, Transformers, etc.)
• Enables multi-task and multi-modal learning
• Includes a suite of visualization tools for model analysis
• Integrates with MLflow for experiment tracking
• Offers a programmatic API and command-line interface

Here‘s an example of what a simple image classification model definition might look like in Ludwig:

input_features:
    - name: image
      type: image
      encoder: resnet

output_features:
    - name: class
      type: category

training:
    optimizer: 
        type: adam
    epochs: 50
    batch_size: 128
    early_stop: 5
    learning_rate: 0.001

This declarative approach to deep learning has several benefits:

• Lowers the barrier to entry for non-experts
• Enables rapid prototyping and iteration
• Reduces boilerplate code and potential for errors
• Allows for easy sharing and reuse of models

Some potential use cases for Ludwig include:

• Text classification and sentiment analysis
• Image recognition and object detection
• Audio classification and speech recognition
• Multi-modal tasks (e.g. visual question answering)

While Ludwig doesn‘t offer the full flexibility and customization of frameworks like TensorFlow or PyTorch, it‘s a valuable tool for quickly building and deploying deep learning models. It will be interesting to see how the project evolves and incorporates more advanced features like AutoML and neural architecture search.

Reddit Discussions

Aside from the technical projects on GitHub, the Reddit data science community is a great place to find thought-provoking discussions and insights from practitioners in the field. Here are a few highlights from February:

Are Difficult Coding Challenges Necessary for Data Science Interviews?

A popular thread debated the merits of using difficult coding challenges (e.g. LeetCode) in data science interviews. Many users argued that these challenges aren‘t representative of actual data science work and place too much emphasis on computer science fundamentals over statistics and domain knowledge.

As one user put it:

"Data science isn‘t about writing the most efficient sorting algorithm. It‘s about understanding the data, asking the right questions, and communicating insights effectively. Coding challenges can be useful for testing baseline programming skills but shouldn‘t be the main focus of the interview."

Others pointed out that the expectation depends on the role and company. For machine learning engineer or data engineer positions, coding challenges may be more relevant. But for more analytics-focused data scientist roles, SQL and practical data manipulation skills are often more important.

The general consensus seemed to be that a mix of coding challenges, take-home assignments, and open-ended discussions is the best approach for evaluating data science candidates. Ultimately, the interview process should align with the actual responsibilities and skills required for the job.

Advice for Data Science Students Feeling Overwhelmed

Another highly upvoted post offered advice for aspiring data scientists who are feeling overwhelmed by the breadth and depth of the field. The author shared their own experience of trying to learn everything at once and burning out.

The top comments emphasized the importance of focusing on fundamentals first before diving into advanced topics or specific tools. Some key pieces of advice:

• Master the basics of statistics, linear algebra, and programming
• Work on practical projects to solidify your understanding
• Don‘t get caught up in the hype of the latest models or frameworks
• Develop a strong understanding of the data science process end-to-end
• Cultivate communication and problem-solving skills
• Be patient and persistent in your learning journey

As one commenter summed it up:

"Data science is a marathon, not a sprint. It‘s easy to get overwhelmed by all the shiny new techniques and tools out there. But at the end of the day, it‘s the fundamental skills that will set you up for success in the long run. Focus on developing those first and the rest will follow."

Conclusion

From state-of-the-art generative models to automated machine learning frameworks, the top data science GitHub repositories in February showcased the incredible progress happening in AI and deep learning. At the same time, the Reddit discussions highlighted the importance of core skills, practical experience, and continuous learning for aspiring and practicing data scientists.

As an AI expert, my key takeaways from this month‘s roundup are:

1. Keep up with the latest research and tools, but don‘t get caught up in the hype. Focus on understanding the fundamental techniques and their implications.

2. Experiment with new models and frameworks, but always consider the ethical implications and potential for misuse. We have a responsibility to develop AI responsibly.

3. Prioritize practical skills and domain knowledge over abstract coding challenges. The best data scientists are able to translate technical expertise into real-world impact.

4. Embrace the open-source community and contribute back where you can. We all benefit from the collective knowledge and efforts of the data science community.

5. Stay curious and never stop learning. The field of AI is evolving rapidly and there‘s always more to explore. But remember to pace yourself and focus on the fundamentals.

I hope this post has given you a taste of the exciting developments in data science and inspired you to dig deeper into some of these projects and discussions. As always, I‘d love to hear your thoughts and feedback in the comments below.

Disclosure: I am not affiliated with any of the projects or companies mentioned in this post. All opinions are my own.