Soket AI Partners Google Cloud To Launch Multilingual AI Model

Indian AI is witnessing a big step forward with the introduction of Pragna-1B. This new initiative is a collaboration between Soket AI Labs, a leading Indian AI research firm, and Google Cloud, the global tech giant. Pragna-1B is a game-changer designed specifically to bridge the language gap in India. As India's first open-source multilingual AI model, Pragna-1B provides developers with cutting-edge Machine Learning (ML) and Natural Language Processing (NLP) capabilities.

Read In Short:

• Soket AI Labs partners with Google Cloud to unveil Pragna-1B, India's first open-source multilingual AI model.
• Pragna-1B provides developers with advanced Multilingual Language Processing (MLP) capabilities, catering to Hindi, English, Bengali, and Gujarati.
• The open-source nature of Pragna-1B fosters collaboration and accelerates the development of Vernacular language AI solutions in India.

👁 Soket-AI-Partners-Google-Cloud-To-Launch-Multilingual-AI-Model

What is Pragna-1B?

Pragna-1B is a first-of-its-kind AI tool for India, built by Soket AI Labs with Google Cloud's help. It's like a super-translator that understands and creates text in Hindi, English, Bengali, and Gujarati. This freely available (open-source) resource empowers developers to create new technology like chatbots and virtual assistants that can understand people in their languages. It's a big step towards making technology accessible to everyone in India.

How Does Pragna-1B Work?

Pragna-1B uses the power of transformers, a deep learning architecture that excels at NLP tasks. It's a decoder-only model, meaning it focuses on generating text based on a given input.

• Input Processing: The user feeds text data in any of the supported languages (Hindi, English, Bengali, or Gujarati) into the model.
• Tokenization: Pragna-1B breaks down the input text into smaller units called tokens. These tokens can be words, characters, or even sub-word units.
• Encoding: The model encodes the tokens using techniques like positional encoding, capturing the meaning and context of each token within the sequence.
• Decoding and Text Generation: Using its internal layers and attention mechanisms, Pragna-1B decodes the encoded representation and generates text that aligns with the input and the chosen language.

How to use Pragna-1B

While Pragna-1B isn't a direct app you can download, it acts as a behind-the-scenes engine for developers. Here's a simplified idea:

Step 1: Developers Get Access

Those building tech tools like chatbots or translation services can access Pragna-1B's open-source code.

Step 2: Integration

They can then integrate Pragna-1B's capabilities into their projects.

Step 3: Power of Languages

This allows their creations to understand and respond in Hindi, English, Bengali, or Gujarati.

So, while you won't directly use Pragna-1B itself, it's the engine behind future AI tools that will understand and speak to you in your preferred Indian language!

Applications of Pragna-1B

Pragna-1B opens doors to a multitude of applications that can revolutionize how we interact with technology in India. Here are some potential applications:

• Machine Translation: Pragna-1B can bridge the language gap by providing accurate and efficient machine translation between Hindi, English, Bengali, and Gujarati. This can empower communication and content accessibility across diverse regions.
• Chatbots and Virtual Assistants: By integrating Pragna-1B, chatbots and virtual assistants can understand and respond to user queries in multiple Indian languages, enhancing user experience and inclusivity.
• Text Summarization and Content Creation: Pragna-1B can be used to generate summaries of factual topics or even create new content in various Indian languages. This can be instrumental in the education and media sectors.
• Sentiment Analysis: Pragna-1B can analyze the sentiment of text data in Indian languages, providing valuable insights for businesses and social media platforms.

Pragna-1B Architecture Overview

1. Transformer-based model (inspired by TinyLlama):

• Layers: 22
• Attention Heads: 32
• Context Length: 2048 tokens
• Hidden Dimension: 2048
• Expansion Dimension: 5632
• Vocabulary Size: 69632

2. Rotary Positional Encoding: uses base 10,000 for positional information.

3. Normalization: RSNorm with epsilon 1e-5.

4. Activation Function: Sigmoid Activation Unit (SiLU).

5. Grouped Query Attention: Improves training speed and memory efficiency, allowing inference on lower-compute devices.

6. Trained on GenAI Studio: Proprietary platform for scaling models across GPUs/accelerators with fault tolerance.

7. Development Tools:

• Triton (OpenAI): Creates high-performance CUDA kernels.
• Fully Sharded Data Parallel (FSDP): Enables distributed training.
• FlashAttention2: Speeds up training and inference

Pragna-1B Data Training

Training Pragna-1B required a special focus because large datasets for Indian languages are rare. Here's what they used:

1. Bhasha: Soket AI Labs created their dataset called Bhasha, translating millions of English Wikipedia articles into Hindi and other Indian languages.
2. Bhasha-wiki-indic: This is a filtered version of Bhasha focusing on content specific to India, helping the model understand Indian culture and context.
3. Bhasha-SFT: This dataset trains the model for various tasks like question answering and conversation, making it more versatile.
4. External Datasets: They also included existing datasets like SlimPajama (mostly English) and Sangraha-Verified (verified data in multiple Indian languages) to further enrich the training process.

Benefits of Open-source Multilingual AI Models Like Pragna-1B

• Faster Innovation: More minds working together means quicker progress in building AI solutions for Indian languages.
• Cost-effective Development: No licensing fees! Open-source models make AI development accessible to a wider range of creators.
• Community Power: Anyone can contribute and improve the model, leading to a stronger overall AI tool.
• Tailored Solutions: Open access allows developers to customize Pragna-1B for specific needs and languages.

Difference Between Pragna-1B and Other Open-Source Multilingual AI Models

Features	Pragna-1B	mBERT	XLM-Roberta
Focus	Indian Languages (Hindi, English, Bengali, Gujarati)	Multilingual (100+ Languages)	Multilingual (100+ Languages)
Model Type	Decoder-only Transformer	Masked Language Model (MLM)	Masked Language Model (MLM)
Open-Source	Yes	Yes	Yes
Strengths	Efficient, Culturally-aware of Indian languages	Versatile, Handles many languages	Versatile, Handles many languages
Best suited for	NLP tasks in Indian languages	General-purpose NLP tasks	General-purpose NLP tasks
Parameter Size	1.25 Billion	137B or 3 Billion	650M or 1.5 Billion

mBERT and XLM-Roberta are powerful models, but they may require more fine-tuning for tasks specific to Indian languages. Pragna-1B's focus on Indian languages and its efficiency make it a strong choice for developers working in that region.

Conclusion

In conclusion, Soket AI Labs and Google Cloud's collaboration on Pragna-1B, India's first open-source multilingual AI model, marks a significant milestone in bridging the language gap. This Machine Learning (ML) marvel empowers developers with Multilingual Language Processing (MLP) capabilities for Hindi, English, Bengali, and Gujarati. Open-sourcing Pragna-1B fosters innovation and paves the way for more inclusive Vernacular language AI solutions across India.