The Sequence Scope: A New Open Source Massive Language Model

Weekly newsletter with over 120,000 subscribers that discusses impactful ML research papers, cool tech releases, the money in AI, and real-life implementations.

📝 Editorial: A New Open Source Massive Language Model

Large language models are the norm of the day in deep learning. Every other month, we see news of a new multi-billion parameter pretrained model reaching new milestones on different language tasks. Despite that progress, only a handful of these models are available to the broader machine learning (ML) research community. The issue is not so much about AI giants trying to be protective about their IP and more about the computational and ethical challenges related to making this type of models readily available. Large language models’ high computational and energy requirements represent a high barrier to entry for most organizations. The ethical concerns related to open-sourcing models that can be used for malicious activities, such as fake news/image generation, are even more critical. Regardless of the challenges, we have seen notable steps toward responsible open-sourcing large language models.