The Sequence Scope: The Race to Improve Reinforcement Learning
Weekly newsletter with over 80,000 subscribers that discusses impactful ML research papers, cool tech releases, the money in AI, and real-life implementations.
The Sequence Scope is a summary of the most important published research papers, released technology and startup news in the AI ecosystem in the last week. This compendium is part of TheSequence newsletter. Data scientists, scholars, and developers from Microsoft Research, Intel Corporation, Linux Foundation AI, Google, Lockheed Martin, Cardiff University, Mellon College of Science, Warsaw University of Technology, Universitat Politècnica de València and other companies and universities are already subscribed to TheSequence.
Subscribe to stay up-to-date with the most relevant projects and research papers in the AI world. Trusted by 85,000+…
📝 Editorial: The Race to Improve Reinforcement Learning
Reinforcement Learning (RL) has been at the center of some of the most important milestones of the last decade of deep learning. DeepMind’s RL-based AlphaGo is considered by many the “Sputnik moment” in artificial intelligence (AI), responsible for sparking an innovation race between the top AI labs in the world. After AlphaGo, RL became sort of a pop-culture term in AI with many research papers making grandiose claims about RL applications that have little correlation with reality. There is something seductive about the idea of learning by trial and error that shares some resemblance with human intelligence. However, the fact is that, despite its popularity, RL techniques remain incredibly challenging and computationally expensive to implement, and most of the applications remain constrained to gaming.
RL applications might not yet be mainstream, but research is accelerating at a frantic pace. Just this week, there were over four major RL papers published by AI labs like Google Research and Berkeley University proposing new methods to improve RL techniques. AI technology incumbents like Microsoft, Amazon, and Google have made RL a centerpiece of their machine learning product strategy. That movement should result in better frameworks and platforms that streamline the implementations of RL applications in the real world. RL might have been at the forefront of some of the most important recent milestones in deep learning but the race is just starting.
If you are interested in RL, stay tuned and subscribe to TheSequence Edge if you haven’t yet because, in the next few weeks, we will be publishing a very extensive series covering RL research and technology. Some RL fun coming your way 😉
🗓 Next week in TheSequence Edge:
Edge#83: the final issue in our N-shot learning mini-series — the concept of One-Shot Learning; Siamese Neural Networks architecture for one-shot-learning models; the review of the ONNX standard.
Edge#84: deep dive into Snorkel Flow — one of the most complete machine learning platforms in the market.
🔎 ML Research
Massive Training Scale
Microsoft Research published a paper unveiling a new technique that uses its DeepSpeed framework to achieve massive levels of training scalability ->read more on Microsoft Research blog
Hyperparameter Optimization and Reinforcement Learning
The famous Berkeley AI Research lab (BAIR) published a paper proposing a method that leverages AutoML to tune hyperparameters in model-based reinforcement learning solutions ->read more on BAIR blog
Improving Reinforcement Learning
Google Research published a paper proposing a new computational graph method to improve the interpretability of reinforcement learning methods ->read more on Google Research blog
Multi-Task Reinforcement Learning for Robotics
Google Research published an amazing paper discussing MT-OPT, a new multi-task reinforcement learning method for training robots at scale ->read more on Google Research blog
Better Reward Functions for Reinforcement Learning
Berkeley AI Research lab (BAIR) published a paper introducing EPIC, a more optimal reward function for reinforcement learning methods ->read more on BAIR blog
🤖 Cool AI Tech Releases
DeepMind open-sourced JAX, a new framework to accelerate deep learning research ->read more on DeepMind blog
Machine Learning for C++
Facebook AI Research(FAIR) open-sourced a new framework for building machine learning models using C++ ->read more on FAIR blog
💸 Money in AI
- Hive, a company providing pre-trained ML models for content moderation, raised $85 million in Series D co-led by Glynn Capital and General Catalyst.
- Big data analytics company Unsupervised raised $35 million in a Series B round led by Cathay Innovation and Signalfire. The platform, built on unsupervised learning for analytics, helps organizations turn the complexity of data into business insights.
- Computer vision startupOrca AI raised $13 million in Series A funding led by OCV Partners. Their AI-powered marine collision avoidance system provides real-time insights for data-driven decisions.
- Real-time information system Applied XL raised $1.5 million in seed funding. Two founders are both ex-Wall Street Journal, in Applied XL they build information systems powered by editorial algorithms that combine the precision of data science with the high standards of journalism.
- Privacy-preserving platform for collaborative data science Cape Privacy raised $20 million in a Series A led by Evolution Equity Partners. On its platform, data scientists can collaborate with multiple parties on model development by using encrypted data.
- Data infrastructure startup Era Software raised $15.25 million in a round led by Playground Global. They created the EraDB time-series database architecture for log management.
- Data management startup CluedIn raised a $15 million Series A funding round led by Dawn Capital. CluedIn streamlines the process of making data ready for insights, without compromising its fidelity or flexibility.
- AI video production startup Synthesia just raised a $12.5 million Series A funding round led by FirstMark Capital. Their promise is that creating a video with them is as easy as writing an email. A personal note from the editor: “I’ve been looking through so many AI startups the last ten months but this one really made me shiver. AI presenters look very real. It’s even eerie. Very impressive.”