Skip to main content

Overview

Welcome to DynamoEnhance, your comprehensive solution for improving Retrieval-Augmented Generation (RAG) pipelines, ensuring robust PII redaction, and training models with differential privacy. Our suite of tools and SDKs is designed to address the most pressing challenges in handling sensitive data and optimizing machine learning workflows.

PII Redaction SDK​

Our PII Redaction SDK offers advanced tools and methods for removing, redacting, and anonymizing personally identifiable information (PII) from text. Unlike traditional approaches that rely on BIO tags, our SDK provides a streamlined solution that integrates state-of-the-art models for accurate and efficient PII classification.

Key Features

  • Streamlined PII Redaction: Our state-of-the-art model intelligently tags the PII inside your text.
  • Efficient Workflows: Tailored methods for accurate PII handling across a range of usecases.
  • Comprehensive Documentation: Walkthroughs and tutorials that showcase the model’s abilities and appropriate usage.

The DynamoEnhance Fine-Tuning SDK provides advanced tools for fine-tuning large language models (LLMs) with differential privacy, ensuring high performance while protecting sensitive data. This is crucial for organizations handling PII and other sensitive information.

Benefits of Fine-Tuning with Differential Privacy​

  1. Enhanced Privacy: Ensures minimal impact of individual data points on the overall training process, reducing data memorization and leakage.
  2. Regulatory Compliance: Helps meet privacy standards and regulations like GDPR and CCPA by providing strong privacy guarantees.
  3. Trust and Security: Builds trust with users and stakeholders by ensuring data security during the training process.
  4. Operationalize Privacy-by-Design: Embed privacy into your organization.

Key Features

  • Differential Privacy Integration: Apply differential privacy during fine-tuning with a single line of code.
  • Flexible Configuration: Use YAML files to customize training parameters, including privacy settings, model parameters, and dataset configurations.
  • Compatibility: Supports integration with popular libraries such as Transformers, HuggingFace Hub, and LoRA (Low-Rank Adaptation).

LLM Evaluation​

The DynamoEnhance SDK offers a comprehensive suite of tools for quantitatively assessing the performance of large language models (LLMs). It includes methods for evaluating text similarity, redundancy, and coverage through metrics like compression ratio, cosine similarity (using BERT and TF-IDF) and n-gram overlap These tools enable detailed analysis of model-generated texts, ensuring that outputs are semantically accurate, concise, and faithful to the reference texts. Additionally, the SDK provides functionalities for assessing the relevance and faithfulness of responses in question-answering systems, enhancing the reliability of information retrieval and conversational AI applications.

The SDK also features methods for generating question-answer pairs from given contexts, supporting diverse sampling strategies to create comprehensive evaluation datasets. The wrapper functions streamline the computation of multiple metrics, simplifying the evaluation process and providing a high level overall view of model performance.

Model Alignment​

The DynamoEnhance Alignment SDK is a comprehensive toolkit designed to enhance the safety and establish robust guardrails around language models. It provides a range of methods that focus on aligning models with specified policies and ensuring that their outputs are free from harmful content. The SDK is divided into alignment functions, which help in tuning models using preference data and reinforcement learning techniques, and helper functions, which support various safety and guardrail use cases.

Key Features

Alignment Functions: These functions assist in aligning language models with specific policies by generating relevant prompts and improving response quality. They leverage diverse and in-domain processes to ensure comprehensive coverage and adherence to guidelines, facilitating the creation of models that produce safe and reliable outputs.

Helper Functions: These functions provide essential support for safety and guardrail use cases. They enable efficient data handling, similarity detection, and response improvement, ensuring that models maintain high standards of safety and compliance. These tools are crucial for developers looking to enhance the robustness and reliability of their language models.