Steal the Code (Ethically) and Get Better at ML/AI Research

As you venture deeper into the exciting world of machine learning and artificial intelligence, you‘ll undoubtedly come across many fascinating research papers detailing novel approaches and state-of-the-art results. But a common frustration is that the code used to produce those results is often missing or not easily accessible.

Fear not aspiring ML/AI researcher! There are many ways to ethically "steal" the code from research papers to accelerate your own learning and work. The key is knowing where to look and how to navigate the evolving norms around open source and reproducibility in ML/AI.

In this post, we‘ll explore the importance of code sharing in research, highlight platforms that make it easier to find papers with code, and share tips for making the most of this valuable resource. By the end, you‘ll be ready to dive in and start replicating, remixing and building upon the latest ML/AI research—all while giving proper credit and contributing to a more open, collaborative field.

Why Open Source Code Matters in ML/AI Research

The incredible progress in machine learning and AI in recent years is thanks in large part to the open source movement and the willingness of researchers to share their code. Making source code publicly available alongside research papers has many benefits:

• Reproducibility: It allows others to verify results and catch errors
• Accessibility: It makes it easier for students and researchers, especially those with less resources, to learn from and build upon state-of-the-art work
• Efficiency: It reduces duplication of effort and accelerates overall progress as researchers can more quickly iterate and innovate
• Collaboration: It facilitates collaboration, discussion and extension of ideas across groups and institutions

As ML/AI increasingly touches every aspect of society, transparency and reproducibility is more critical than ever. Public code helps promote accountability and trust in a field that can often seem like a black box.

While the ML/AI research community has made great strides in open sourcing code, there‘s still room for improvement. A 2022 study found that only around 25% of ML papers consistently link to their source code. High-profile conferences like NeurIPS now encourage code submission and the NSF is piloting reproducibility requirements for grant-funded AI research.

As a researcher, making your own code available and utilizing shared code from others in an ethical manner are key ways you can contribute to a stronger, more robust ML/AI ecosystem. Let‘s look at some top resources to get you started.

Finding ML/AI Papers with Code

There are now a number of excellent platforms designed to make it quick and easy to find papers with available source code in different areas of ML/AI. Some top ones to explore:

Papers With Code

The largest free and open resource of ML papers with code, linking to over 80,000 papers with 150,000+ implementations across every imaginable application. Features:

• Browse state-of-the-art papers by task, dataset, method
• Quickly compare code and reported results across papers
• Discover trending research areas
• Filter by framework (PyTorch, Tensorflow, etc) and domain (CV, NLP, RL, etc)
• Contribute by adding links to papers and code on GitHub

AI Code Datasets

Curated collections of papers focused on applied ML/AI with available source code:

• CodeNet: 13.8 million code samples in 55+ languages from ML/AI papers assembled by IBM
• CodeSearchNet: 2 million code samples from papers by task assembled by GitHub
• CoNaLa: Database of 600,000 NL-code pairs mined from papers and docs online

ML/AI Model Zoos

Model zoos package up trained ML/AI models from research papers along with inference code, training code, and examples to try them out:

• Tensorflow Model Zoo
• PyTorch Model Zoo
• Hugging Face Model Hub
• ModelDepot.io
• OpenMMLab

As you can see, there are now rich resources available to find top papers and get your hands on the code to really dig in, learn the methods and reproduce results yourself. Next we‘ll cover tips for making the most of research code.

Tips for Understanding and Extending ML/AI Research Code

Digging into cutting-edge ML/AI research code bases can seem daunting at first, especially for students and those newer to the field. Have no fear! Here are some tips for getting the most of out of the experience:

1. Read the whole paper closely first. Make sure you have a solid grasp of the key ideas, methods, diagrams, and results. Take notes and write down your questions. The code will make a lot more sense with this context.

2. Run the code and reproduce the results. Clone the repo, follow the setup instructions, and run the core scripts or notebook to replicate key findings from the paper. Consult the README, code comments, and any documentation. Compare your outputs to what‘s reported in the paper.

3. Inspect the code structure and core components. Get a high-level view of how the code base is organized, the key classes/functions, and how data flows through the pipeline. Identify the novel parts that implement the main ideas from the paper vs the scaffolding.

4. Experiment and extend the code. Swap in your own dataset, try different model architectures or hyperparameters, implement a new feature or performance tweak. Use logging and visualization to compare your outputs. Brainstorm ideas for novel extensions to follow up on.

5. Discuss and get help from the community. Post questions, insights and experiments on Twitter tagging the paper authors. Join the ML/AI Discord, Reddit or Slack for that domain. Find the project on GitHub and browse issues and discussions. If you get stuck, don‘t be afraid to reach out to the authors directly via email for pointers.

6. Contribute back your code and results. If you identify an issue or improve on the code, submit a pull request to the repo. Write up your build and tag the authors for feedback. Consider publishing your own follow-up paper open sourcing your new code. Pay it forward to keep the virtuous open source research cycle going!

By being proactive, persistent and connecting with the community, you can rapidly accelerate your ML/AI skills by learning from the best. Diving deep into research code is challenging but highly rewarding.

Advancing Reproducibility in ML/AI

As an ML/AI practitioner, there are a number of ways you can contribute to improving reproducibility and accelerating research:

• Set a positive example by always releasing your code, models, and data alongside your own papers and articles
• Support and engage with reproducibility initiatives like the ML Reproducibility Challenge, Papers With Code, and reproducibility checklists/guidelines
• Participate in reproducibility tracks at major conferences like ICML, AAAI, and NeurIPS to get recognized for your efforts
• Advocate for stronger reproducibility standards at your institution or company, such as making sure internal reports and papers have code
• Volunteer to review for journals and conferences to help catch issues related to code release and reproducibility
• When using code from others‘ research, give proper attribution, citation and credit—and let the authors know if you build on their work

Conclusion: Steal the Code, Share the Knowledge

Research progress in the fast-moving fields of machine learning and artificial intelligence relies on an open, collaborative approach. By embracing the current culture of sharing code alongside publications—and directly contributing to it—we can all play a role in accelerating innovation and impact.

Seek out every opportunity to learn from and build upon the work of others. Always strive to share your own work just as openly. Together, we can steal the code (ethically) and create a more vibrant, reproducible, and accessible future for ML/AI research.