Soket AI Partners Google Cloud To Launch Multilingual AI Model

Indian AI is witnessing a big step forward with the introduction of Pragna-1B. This new initiative is a collaboration between Soket AI Labs, a leading Indian AI research firm, and Google Cloud, the global tech giant. Pragna-1B is a game-changer designed specifically to bridge the language gap in India. As India’s first open-source multilingual AI model, Pragna-1B provides developers with cutting-edge Machine Learning (ML) and Natural Language Processing (NLP) capabilities.

Read In Short:

• Soket AI Labs partners with Google Cloud to unveil Pragna-1B, India’s first open-source multilingual AI model.
• Pragna-1B provides developers with advanced Multilingual Language Processing (MLP) capabilities, catering to Hindi, English, Bengali, and Gujarati.
• The open-source nature of Pragna-1B fosters collaboration and accelerates the development of Vernacular language AI solutions in India.

Pragna-1B is a first-of-its-kind AI tool for India, built by Soket AI Labs with Google Cloud’s help. It’s like a super-translator that understands and creates text in Hindi, English, Bengali, and Gujarati. This freely available (open-source) resource empowers developers to create new technology like chatbots and virtual assistants that can understand people in their languages. It’s a big step towards making technology accessible to everyone in India.

Pragna-1B uses the power of transformers, a deep learning architecture that excels at NLP tasks. It’s a decoder-only model, meaning it focuses on generating text based on a given input.

• Input Processing: The user feeds text data in any of the supported languages (Hindi, English, Bengali, or Gujarati) into the model.
• Tokenization: Pragna-1B breaks down the input text into smaller units called tokens. These tokens can be words, characters, or even sub-word units.
• Encoding: The model encodes the tokens using techniques like positional encoding, capturing the meaning and context of each token within the sequence.
• Decoding and Text Generation: Using its internal layers and attention mechanisms, Pragna-1B decodes the encoded representation and generates text that aligns with the input and the chosen language.

While Pragna-1B isn’t a direct app you can download, it acts as a behind-the-scenes engine for developers. Here’s a simplified idea:

Step 1: Developers Get Access

Those building tech tools like chatbots or translation services can access Pragna-1B’s open-source code.

Step 2: Integration

They can then integrate Pragna-1B’s capabilities into their projects.

Step 3: Power of Languages

This allows their creations to understand and respond in Hindi, English, Bengali, or Gujarati.

So, while you won’t directly use Pragna-1B itself, it’s the engine behind future AI tools that will understand and speak to you in your preferred Indian language!

Pragna-1B opens doors to a multitude of applications that can revolutionize how we interact with technology in India. Here are some potential applications:

• Machine Translation: Pragna-1B can bridge the language gap by providing accurate and efficient machine translation between Hindi, English, Bengali, and Gujarati. This can empower communication and content accessibility across diverse regions.
• Chatbots and Virtual Assistants: By integrating Pragna-1B, chatbots and virtual assistants can understand and respond to user queries in multiple Indian languages, enhancing user experience and inclusivity.
• Text Summarization and Content Creation: Pragna-1B can be used to generate summaries of factual topics or even create new content in various Indian languages. This can be instrumental in the education and media sectors.
• Sentiment Analysis: Pragna-1B can analyze the sentiment of text data in Indian languages, providing valuable insights for businesses and social media platforms.

1. Transformer-based model (inspired by TinyLlama):

• Layers: 22
• Attention Heads: 32
• Context Length: 2048 tokens
• Hidden Dimension: 2048
• Expansion Dimension: 5632
• Vocabulary Size: 69632

2. Rotary Positional Encoding: uses base 10,000 for positional information.

3. Normalization: RSNorm with epsilon 1e-5.

4. Activation Function: Sigmoid Activation Unit (SiLU).

5. Grouped Query Attention: Improves training speed and memory efficiency, allowing inference on lower-compute devices.

6. Trained on GenAI Studio: Proprietary platform for scaling models across GPUs/accelerators with fault tolerance.

7. Development Tools:

• Triton (OpenAI): Creates high-performance CUDA kernels.
• Fully Sharded Data Parallel (FSDP): Enables distributed training.
• FlashAttention2: Speeds up training and inference

Training Pragna-1B required a special focus because large datasets for Indian languages are rare. Here’s what they used:

1. Bhasha: Soket AI Labs created their dataset called Bhasha, translating millions of English Wikipedia articles into Hindi and other Indian languages.
2. Bhasha-wiki-indic: This is a filtered version of Bhasha focusing on content specific to India, helping the model understand Indian culture and context.
3. Bhasha-SFT: This dataset trains the model for various tasks like question answering and conversation, making it more versatile.
4. External Datasets: They also included existing datasets like SlimPajama (mostly English) and Sangraha-Verified (verified data in multiple Indian languages) to further enrich the training process.

• Faster Innovation: More minds working together means quicker progress in building AI solutions for Indian languages.
• Cost-effective Development: No licensing fees! Open-source models make AI development accessible to a wider range of creators.
• Community Power: Anyone can contribute and improve the model, leading to a stronger overall AI tool.
• Tailored Solutions: Open access allows developers to customize Pragna-1B for specific needs and languages.

mBERT and XLM-Roberta are powerful models, but they may require more fine-tuning for tasks specific to Indian languages. Pragna-1B’s focus on Indian languages and its efficiency make it a strong choice for developers working in that region.

In conclusion, Soket AI Labs and Google Cloud’s collaboration on Pragna-1B, India’s first open-source multilingual AI model, marks a significant milestone in bridging the language gap. This Machine Learning (ML) marvel empowers developers with Multilingual Language Processing (MLP) capabilities for Hindi, English, Bengali, and Gujarati. Open-sourcing Pragna-1B fosters innovation and paves the way for more inclusive Vernacular language AI solutions across India.

Can I directly use Pragna-1B like a regular app?

No, Pragna-1B is an open-source engine for developers to build AI tools that understand Indian languages.

What kind of AI tools can be built with Pragna-1B?

Chatbots, virtual assistants, machine translation services, and text analysis tools are some possibilities.

Does Pragna-1B understand all Indian languages?

Currently, Pragna-1B focuses on Hindi, English, Bengali, and Gujarati, but future versions may include more languages.

What are the advantages of open-source multilingual AI models?

Faster innovation, cost-effective development, community-driven improvement, and customization for specific languages.

How is Pragna-1B different from other multilingual AI models?

Pragna-1B is specifically designed for Indian languages, efficient, and culturally aware, while other models may require more data or be broader in language coverage.