It’s been nearly a year since the release of the original LLLaMA model, and the landscape of open language models has seen significant evolution since then. From the initial semi-open license of LLLaMA to the subsequent flood of open LLMs like Alpaca and Vicuna, the field has been rapidly expanding. The release of LLLaMA 2 marked a pivotal moment, offering a more open-source approach that empowered users to build and innovate. Later, models from Mistral, including Mistral 7B and Mixtral, further diversified the landscape.
Google Steps In: Gemma
Despite Google’s history of open-sourcing models like BERT and T5, the scene had been relatively quiet until now. Enter Gemma, Google’s latest offering, comprising a suite of models aimed at democratizing access to state-of-the-art language models.
Introducing Gemma Models
Gemma comes in four variants: a 7 billion base model, a 7 billion instructor model, a 2 billion base model, and a 2 billion instructor model. These models, derived from the same technology as Gemini, are text-to-text decoder-only models available in English, with open weights.
The Power of Gemma
One standout feature of Gemma models is their training on a staggering 6 trillion tokens, surpassing previous benchmarks in token count for open-style models. This extensive training corpus includes web documents, code, and mathematical data, meticulously processed to remove sensitive information.
Getting Started with Gemma
Accessing Gemma models involves a straightforward process of filling out an access request form. Once granted access, users can utilize the models either directly on platforms like Kaggle or by downloading the weights for external use, such as in Colab.
Using Gemma with Keras NLP
For those keen to dive into Gemma, using Keras NLP provides a convenient starting point. With Keras 3.0 and the Keras NLP library, users can easily interact with Gemma models, exploring functionalities like question answering and batch processing.
Looking Ahead
While Gemma’s release marks an exciting development, the true potential lies in the community’s hands. Expect to see innovative applications and fine-tuned models emerging in the coming days and weeks. Stay tuned for future updates as we delve deeper into Gemma and its capabilities.
Final Thoughts
As we embark on this journey with Gemma, your feedback and questions are invaluable. Drop them in the comments below, and don’t forget to like and subscribe for more content. Until next time, happy modeling!