# Data and Knowledge Access
Source: https://docs.ekyam.ai/api-reference/knowledgeaccess
### **• Understanding the Core Connection**
The Chronicle Data is one of the central and inevitable ledgers within Ekyam. These agents are responsible for recording (reading) new events and retrieving past data events (writing).
**How do these Agents interact with Core Chronicle Data**
AI Agents interact with the Core Chronicle Data via a secure, two-part system: Writing new data and Reading it as well. This interaction is governed by a read-only model via MCP server and the Knowledge Graph is exposed via the tool to it.
**→ Reading from Chronicle:** Agents use the knowledge graph for performing complex queries. Since the Knowledge Graph is an interconnected representation of the data in the chronicle, the agent can query the graph for relationships instead of searching several log events.
A key component of the retrieval process, The Knowledge Graph, is hosted on Neo4j on Google Cloud, which requires valid credentials and security protocols for access.
### **• Querying Retail Knowledge Graph**
A Retail Knowledge Graph is a repository that connects the key retail entities like products, customers, stores and suppliers in a semantic network. The agents interact with the RKG to follow a multi-hop reasoning with a chain of relationships to gather complex and rich information.
This showcases the role of an AI agent from a search tool to an intelligent response engine.
**Demonstrating Agent-Graph Interaction**
Agents query the RKG using Cypher queries that traverse multiple relationships types ((eg. MATCH (c:Customer)\[:PURCHASED]> (p:Product) \[:BELONGS\_TO] (cat: Category)). This enables them to retrieve contextual information like customer preferences, product associations, and purchase patterns in a single query rather than multiple database calls.
**Examples of Complex Queries**
Take a look at below mentioned examples of complex queries that leverage the graph to answer high-level business questions:
→**Customer segmentation: MATCH**
(c:Customer)\[PURCHASED]>(p: Product)\[:BELONGS\_TO]>(cat: Category) WHERE [cat.name](http://cat.name) IN \['Premium', 'Luxury'] RETURN c.demographic, COUNT(DISTINCT p) as purchase\_count ORDER BY purchase\_count DESC - identifies high-value customers by their premium product purchases.
→ **Cross-Selling Opportunities: MATCH**
(c:Customer)\[:PURCHASED]>(p1:Product)\[:FREQUENTLY\_BOUGHT\_WITH]>(p2:Product) WHERE NOT(c)\[:PURCHASED]>(p2) RETURN [c.id](http://c.id), [p1.name](http://p1.name), [p2.name](http://p2.name) - finds customers who bought one product but not its frequently co-purchased companion.
→ **Seasonal Trend Analysis: MATCH**
(p:Product)\[:SOLD\_IN]> (t:Transaction)[:OCCURRED\_IN](s:Season) WHERE [s.name](http://s.name)= 'Holiday' RETURN p.category, AVG(t.quantity) as avg\_quantity, SUM(t.revenue) as total\_revenue - analyzes holiday season performance across product categories.
# Batch Integration
Source: https://docs.ekyam.ai/capabilities/batch-integration
**Ekyam’s Batch Integration for Robust Scheduled Operations**
**Business Case: Strategic Data Processing**
* **Syncing with Legacy Systems**: Several ERPs are designed to generate reports or export data on a scheduled basis (nightly reports). With batch integration, it becomes easy to consume these outputs, thereby allowing modern systems to integrate with and leverage data from these less agile platforms.
* **Bulk Product Catalog Uploads:** A batch upload of adding a large number of new products or making extensive updates to an existing product catalogue is more efficient than real-time updates. This allows for validation and processing of the entire dataset before it is made live.
**Contrast with Real-Time: The Problem of Data Latency**
The fundamental difference between batch and real-time integration lies in **data latency**.
* **Real-time integration** has near-zero latency, as data is delivered when it is created or updated to give instant inventory updates, prevent overselling and fulfill orders.
* **Batch integration**, inherits data latency as data is collected over a period, processed, and then delivered.
**Technical Methods: Scheduled Data Movement**
For implementing batch integration, it involves scheduled data transfers using established protocols:
**→ Scheduled FTP/SFTP File Transfers:** Being a common method for batch processing, you need to Source systems export data into files (e.g., CSV, XML, JSON) and place them on an FTP (File Transfer Protocol) or SFTP (Secure File Transfer Protocol) server. The consuming system connects to this server, downloads the files, processes the data and imports it. SFTP is a more preferred method due to its encryption.
**→ Scheduled Database Queries:** Batch integration involves scheduled jobs that execute SQL queries to extract large datasets from a source database. This extracted data is then loaded into a destination database, or an application reads it directly from the query results for processing. This method is often used for data warehousing or ETL (Extract, Transform, Load) processes.
# Developer Program
Source: https://docs.ekyam.ai/capabilities/developer-program
A Visionary Guide to the Ekyam SDK
### **• What is the Ekyam SDK?**
Ekyam’s SDK is the primary interface for developers to access, control and extend the Ekyam Retail data. It allows the developers to not only just build connections but add new layers of intelligence and automation.
Ekyam’s SDK is a primary Python-based framework for developers to create custom connectors. These connectors allow Ekyam to integrate with and access data from virtually any system, including proprietary platforms, thereby extending Ekyam’s “connect anything”. Additionally, there is a strategic roadmap to release native SDK’s for other key enterprise and key web technology stacks, including JAVA and GO to support a wider developer ecosystem.
### **• Purpose of Ekyam’s SDK**
The three core development pillars for Ekyam’s SDK:
1. **Build Custom Connectors:** The custom connectors will be able to seamlessly integrate any proprietary or niche data source (ERP, warehouse system, or a marketing tool) into the Ekyam system.
**For instance:** A retailer uses a proprietary system to manage custom orders and the system that the retailer uses has no API. A developer uses the Ekyam SDK to build a custom connector that reads data directly from this system's database, and transforms it into a standard Ekyam Data Standards. These high-value custom orders then seamlessly flow into the ERP and fulfillment workflows.
2. **Create Intelligent Automations:** The custom connectors are designed by developers to execute complex workflows, thereby leveraging the full context of Retail Knowledge Graph (RKG) and Universal Ledger to build custom logics.
**For instance:** A developer uses the Ekyam SDK’s **WorkFlowManager** to create a “Smart Returns” process. When a return is initiated, the workflow programmatically checks the customer’s lifetime value with the help of RKG, analyzes the return reason using the AIEnrichmentClient, and checks for potential frauds.
3. **Develop Standalone Retail Applications:** The developers will build entirely new applications (advance analytics dashboards, custom merchandising tools etc) that use Ekyam as their intelligent back-end for real-time, unified retail data.
**For instance:** A merchandising team wants a real-time “Product Performance Dashboard” that uses the sales data from Shopify, POS Systems, and Amazon with inventory data from their WMS and marketing data from CRM. A developer builds a stand-alone web application that uses Ekyam SDK to query the Universal Ledger and RKG. The app provides a unified, live view of product performance that would be impossible to build without Ekyam’s centralized data.
In addition to the above pillars, Ekyam’s SDK includes a **“Retail Transformation Toolkit”,** which is specifically designed to perform data tasks in retail. This kit includes functionalities for:
* **Data Normalization:** Standardize disparate data formats into a consistent schema.
* **Data Validation:** Implement rules to ensure data quality and integrity.
* **Code and Unit Conversions:** Assists in converting product codes, measurement units or currency formats.
### **• Core Components of the SDK**
These are the building blocks that the developer will be using for developing Ekyam connectors.
1. **SecureConnectionManager:** It handles the secure storage and management of credentials that are needed to connect to external systems. A developer declares the authentication type ( OAuth 2.0, API Key). This component efficiently handles the complex token refresh loops, signature generation and secure vaulting so credentials are never exposed in the connector code.
**For instance:** On connecting to Shopify, enter the API key into Ekyam. This stores the key securely, so the custom connector can access Shopify without exposing the password in the code.
2. **DataMapper (Reader and Writer Engine):**
This component translates data from any external format to the Ekyam Internal Data standards. A DataMapper defines a rule, ensuring that the data is understood by the system each time. A developer may use the **DataMapper** to define how the data from a custom source needs to be read and understood by Ekyam; and how Ekyam’s standard data should be written back to the system in its required format.
3. **Schema and Ontology Toolkit:**
This toolkit works with the DataMapper. Developers use this to ensure the data their custom application reads or writes is perfectly structured to become part of the Ekyam Retail Knowledge Graph. The DataMapper translates data to be used by Retail Knowledge Graph.
4. **RetailTransformationEngine:** The developers can use these helpers during the mapping process to add intelligence. It can be used for:
* **Hierarchical Category Mapping:** This is used for intelligently mapping a flat category string from a legacy system to Ekyam’s structured multi-level category model.
* **Sentiment Analysis:** This analysis takes the unstructured text like a review or support email to return a structured response.
5. **AIEnrichmentClient:**
This component provides a direct interface for developers to send data to Ekyam’s AI services for processing, allowing the custom connectors or applications to think beyond simple data transformation. For instance: If a developer wants to send a list of basic products to the client, it can use Ekyam’s AI to write a description. Also, a developer can pass unstructured customer review text to the client, and the AI can analyze this and return with a structured object. The new structured data can then be used for advanced analytics.
6. **WorkflowManager:**
This component allows developers to create, trigger and manage Ekyam Workflows. A custom application can be used to build a new automation process to trigger an existing workflow with a specific data payload.
7. **EventLogClient:**
This component gives the access to the developer to build custom monitoring dashboards or external alerting systems to track integrations and provides a record of every transaction that flows through the platform.
8. **ExceptionhandlingFramework:**\
It is a crucial component for building connectors. Ekyam’s SDK provides a set of specific, pre-defined exception classes (eg, Authentication Error, Data Validation Error). When a developer raises these exceptions, the Ekyam platform can intelligently handle the failure by initiating an automatic retry or creating a detailed alert for the user.
### **• Why build with the Ekyam SDK?**
This section will help the users to understand the advantages of using Ekyam SDK that differentiates it from the generic tools or direct API interaction.
The Ekyam SDK is necessary for developers to integrate a new data source directly into Ekyam’s core systems: Universal Ledger (for a unified data view), AI Engine (for Intelligent enrichment and analysis), and Workflow engines (for automated processes). This capability is especially beneficial for connecting to systems like proprietary internal ERPs, integrating with SaaS applications, and handling a complex custom authentication flow.
By building a custom connector, it will enable in making the data available as a new Queryable Resource for the Ekyam Agentic AI world. As a queryable resource, the RKG (Retail Knowledge Graph)- transforms into structured knowledge that Ekyam’s AI agents can understand and process.
Let us see the why having Ekyam SDK is necessary:
1. **Access a Unified View of Retail Data:** The SDK provides a simple interface to the Ekyam Universal Ledger. This implies that the application can instantly access standardized real-time data from across the entire retail landscape (ERP, WMS, PIM, OMS, CRM, IDocs, EDI etc), without handling the complexity of connecting to each source individually.
**For instance:** A custom application needs to display the Available-to-Promise (ATP) inventory for a product. Instead of writing a code to connect to WMS (On-hand stock), the OMS (committed stock) and ERP (Incoming stock); the developer can make a single, high-level SDK call (sdk.inventory.get\_atp(sku="XYZ-123”). The SDK will then be able to handle the multi-system data aggregation.
2. **Leveraging the Retail Knowledge Graph (RKG):** This is an intelligent semantic model that is used by Ekyam SDK. The SDK provides high-level objects and methods to query the RKG, allowing the users to understand the relationships and meaning between products, inventory, orders, and customers in a simplified manner than using a complex or a custom logic.
**For instance:** If a user wants to build a “Complete the look” feature in the app, the user may use the SDK to ask a semantic question: (sdk.rkg.find\_related\_products(sku="XYZ-123", relationship="Complements"). The RKG understands the relationship between items and returns a list of matching items, which showcases opportunities beyond a simple data retrieval.
3. **Directly integrating with the AI and Agentic Framework:** Ekyam’s SDK methods allows the custom code to directly invoke Ekyam's AI ecosystem. This ecosystem is built on a technical stack that includes MCP, Langchain, Multiple LLMs, and the Ekyam RKG- these are designed with a natural language prompt with a retail-specific context. The custom components become a part of the Ekyam’s AI agents.
**For instance:** The internal dashboard needs to provide a quick summary of performance. A developer can use the SDK to send the Natural Language Query directly to Ekyam’s AI. The AI framework will then handle the complex data retrieval, analysis and human-readable summary.
4. **Build Custom Connectors to Unify the Entire Ecosystem:** The Ekyam SDK is the primary tool for integrating diverse and proprietary systems. Building a connector transforms any data source—from legacy to modern—into intelligent, actionable data within the Ekyam platform.
**For instance:** A custom clienteling app's trapped customer preferences become queryable RKG data via Ekyam’s SDK connector. This enables new capabilities, such as using in-store preferences to power personalized email campaigns through marketing automation tools.
### **• The Custom Connector Lifecycle**
The Ekyam SDK provides a structured lifecycle for building, testing and deploying production-ready custom connectors. This guide will give the users a walk-through of the SDK and also the role of the developer.
The Ekyam SDK provides a structured framework through a Python base class (BaseConnector). By building within this framework, the developer needs to ensure that the custom logic integrates with the Ekyam’s platform’s security, monitoring, workflow, and AI capabilities. The developer can write the code to the external system and the SDK handles the rest of the part.
The first step is to create an organized project structure and the Ekyam Command Line (CLI) makes it simple.\
**Action:** Start by running the Ekyam CLI command with the **Connector** flag to specify you are building a connector:
```
ekyam-cli init --connector "My-Custom-ERP"
```
1. This command generates a boilerplate project with all the necessary files, including [connector.py](http://connector.py) for the logic and [config.py](http://config.py) for settings.
2. The developer will edit the [config.py](http://config.py) file
3. Define the UI fields that a user can see when the connector is set. For example, if your connector needs an API Key and a Server Address, you define them here using a simple Pydantic model.
```
# In config.py
from pydantic import BaseModel, Field, SecretStr
class MyCustomERPConfig(BaseModel):
server_address: str = Field(
...,
title="Server Address",
description="The full URL of the ERP API. e.g., https://api.my-erp.com"
)
api_key: SecretStr = Field(
...,
title="API Key",
description="Your unique API key for authenticating with the ERP."
)
```
*This step will automatically build the user interface for the connector within the Ekyam Platform. Once the configuration is defined, the user needs to ensure that any credentials (like API key or passwords) are handled by Ekyam’s secure vault and are not hardcoded in the connector’s logic.*
The read () method’s job is to read data from the external system and bring it into Ekyam. This is where the logic is implemented to fetch information.
→ In your [connector.py](http://connector.py) file, you will implement the read () method.
Inside this method, the Python code will be:
1. Use the Ekyam SDK’s **SecureConnectionManager** to access the user credentials securely.
2. Make an authenticated call to the external system’s API using a library like **requests** or **httpx**.
3. Use the Ekyam SDK’s **DataMapper** and **RetailTransformationEngine** to convert the raw response into a standardized object (Like an **EkyamOrder**)
4. Return the standardized Ekyam object. The Ekyam platform then ingests this object into the Universal Ledger and makes it available to workflows and the AI engine.
\
**For instance:** Imagine you are connecting to a client’s inventory portal. Your read () method will contain the Python code to log in (using the credentials the user provided by the **SecureConnectionManager**) and fetch the latest inventory. Before returning the data, you have to use the **DataMapper** to translate their field names (e.g., 'stock\_qty') to Ekyam's ('quantityOnHand').
The write () method’s job is the inverse of read (): It takes a standardized Ekyam Object and writes it to the external system.
→ In the [connector.py](http://connector.py) file, implement the write () method.
In this, the Ekyam’s platform will call this method when a workflow needs to send data to your custom- connected system. The code will:
* Receive a standardized Ekyam object as an argument ( e.g., an **EkyamShipment** object)
* Use the DataMapper to transform this object into the specific format the destination API expects.
* Make the authenticated API call to send the formatted data to the external system.
\
**For instance:** When Ekyam needs to send a new Purchase Order to the custom-connected supplier system, it calls the write () method with a standard **EkyamOrder** object. The code will use the DataMapper to transform this object into a specific XML format the client’s portal requires and then make the API call to post it.
Ekyam’s SDK provides an error handling platform.
→ Within the read () and write () methods, wrap the external API calls in try…except blocks.
This allows the developer to catch generic errors (like network timeout) and raise a specific and meaningful exception from the SDK’s **ExceptionHandlingFramework**.
```
# In your read() or write() method
from ekyam_sdk import exceptions
import requests
try:
response = requests.get(url, headers=auth_headers, timeout=10)
response.raise_for_status() # Raises an exception for bad status codes (4xx or 5xx)
except requests.exceptions.HTTPError as e:
if e.response.status_code == 401:
raise exceptions.AuthenticationError("Invalid API Key provided.")
elif e.response.status_code == 429:
raise exceptions.RateLimitError("External API rate limit exceeded.")
else:
raise exceptions.ApiError(f"API Error: {e.response.text}")
except requests.exceptions.RequestException as e:
raise exceptions.ApiConnectionError(f"Connection failed: {e}")
```
This is important because by raising these specific exceptions, you will allow the Ekyam platform to handle the failure intelligently- for instance– by automatically retrying the action in case of a temporary RateLimitError or by creating a detailed alert for the user in case of a persistent AuthenticationError.
Before deploying, it is essential to ensure that the connector is working perfectly. The Ekyam SDK provides a local test harness to simulate the platform environment.
→ Run the CLI command **ekyam-cli test my-connector.**
This command creates a local sandbox that simulates the Ekyam Platform. It allows the developer to:
1. Test read() and write () methods with sample data
2. Verify whether the data mappings and transformations are correct.
3. Check that the connector raises SDK exceptions under failure conditions.
4. The above step is pivotal for building a reliable connector.
Once the connector is developed and tested, the final step is to deploy it.
→ Run **ekyam-cli package my-connector** to validate and bundle your code into a secure, versioned archive.
→ Run **ekyam-cli publish**
**Review Process**: The connector is submitted to the Ekyam team for a security and performance review. This ensures that the apps are safe and secure.
Once the connector is approved, the visibility of the connector can be set to:
* **Private:** The connector will be visible and installable by users within the Ekyam Organization.
* **Public:** The connector will be listed in the public Ekyam platform, making it available to different clients.
# Integration
Source: https://docs.ekyam.ai/capabilities/integration
### **•** **Introduction**
Let us get started with the Ekyam Integration document where the user will learn about the Environment set up, Authentication Methods, and Prerequisites (essential credentials and configurations the team needs from the client to initiate and maintain secure data flows). The document will also explain about a few use cases that weave an easy understanding of the integration.
### **• Basic Environment Setup**
A robust environment setup is critical for consistent development, effective testing, and reliable deployment. This section will highlight the key requirements for each of Ekyam environments: Development (Dev), Demonstration (Demo), and Production (Prod).
**Developer**\
A Dev environment is created for individual developers to write, test and debug code. It also allows the developers to create and test new functionalities and features.
**Staging**\
A Staging environment is a pre-production environment designed for final, comprehensive testing and validation before deploying code to the Production environment. The code deployed to Staging is considered a "release candidate”, implying that it has already passed QA checks.
**Demo**\
It is a client-facing space where the application’s “good-to-go” code is deployed. This implies that the code deployed here has already passed initial development and quality assurance (QA) checks. This environment shows the product’s features and functionalities specifically designed for clients, prospects or external stakeholders.
**Production**\
The Production environment is the live, client-facing system where the application's fully validated and "good-to-go" features and functionalities are deployed. Changes to the code in this environment are subject to **extremely rigorous control and approval processes.**
### **• Authentication Methods**
The choice of authentication methods depends on the specific system being integrated.
* **API key**: Ekyam uses a unique key that is issued by one system to another. The client system includes this key in its request to the server API.
* **OAuth 2.0** (Authorization Framework for Delegated Access): Ekyam uses OAuth 2.0 JWT **authorization framework** that allows a user to grant the client **limited access** to their resources on a service without sharing their login credentials with the third-party app. It does this by issuing access tokens.
If Ekyam needs to access data on Shopify that belongs to a user, OAuth2.0 provides a secure way for the user to grant Ekyam permission. Ekyam gets an access token, and not the user’s password.
* **System Authentication to Hit the API**: While OAuth 2.0 enables a client system to hit an API, it is doing so on behalf of a user or on its own behalf (using client credentials).
* **Two-Factor Authentication (2FA**): Ekyam AI implements the 2FA, prioritizing security and preventing unauthorized access. The 2FA adds an extra layer of security beyond just a password. The users need to provide two different factors of authentication to verify the identity before gaining access to the Ekyam system. This significantly reduces the risk of unauthorized access even if a password is stolen or compromised.
**Let us see a use case for 2FA:**
- Client navigates to the Ekyam login page.
- Enter **registered username** (e.g., [xyz.retail@example.com](mailto:xyz.retail@example.com)) and **password**.
- Click "Login."
* Instead of immediately logging in, the Ekyam system detects that 2FA is enabled for the client’s account.
* **Ekyam sends a unique, time-sensitive verification code** (e.g., a 6-digit number like 12345) to the **registered mobile phone number via SMS**.
* A prompt appears on the Ekyam login screen, asking the client to "Enter the 6-digit code sent to your mobile phone."
* Client receives the SMS code on her phone.
* Enter the code into the Ekyam login screen's prompt.
* The Ekyam system verifies that the entered code matches the one it sent and that it's still valid (i.e., hasn't expired).
* Upon successful verification, the client is granted full access to the Ekyam dashboard.
* **SSO (Single-sign on)**: Ekyam uses a Single Sign-On (SSO) critical feature for its clients. It is an authentication method that allows a user to log in with a single ID and password and then automatically gain access to multiple systems, without requiring to re-authenticate for each one. Essentially, it implies that if the user logs in once, they can access all connected applications.
### **• First Integration Walkthrough: ERP + Shopify + 3PL**
This integration establishes a powerful, automated backbone for Ekyam’s retail operations, ensuring seamless data flow. The integration will give the users an understanding on how an online purchase on Shopify through ERP and fulfillment by 3PL, accurately updates inventory and synchronizes across all systems.
**Systems Involved**
* **ERP (Enterprise Resource Planning):** The central Source of Truth for product master data, inventory levels, sales orders, customer information, and financial records (e.g., SAP Business One, NetSuite).
* **Shopify:** An e-commerce storefront, handling online sales, customer interactions, and product display.
* **3PL (Third-Party Logistics):** Ekyam’s warehouse and fulfillment partner, responsible for storing inventory, picking, packing, and shipping orders.
### **• The Integration Flow (Step-by-Step Data Journey)**
* **What:** Product details (SKU, name, description, price, images, variants), and crucially, **available inventory levels**.
* **Direction:** **ERP ➔ Shopify**
* **Why:** ERP is the single source of truth for products and their quantities. This ensures that what customers see available on Shopify stores is accurate, preventing overselling or stockouts. This typically runs continuously or on a near-real-time schedule.
* **What:** Newly placed sales orders (customer details, line items, quantities, pricing, shipping address, payment status), and details of new customers.
* **Direction:** **Shopify ➔ ERP**
* **Why:** Every online sale needs to be recorded in the ERP for financial tracking, order management, and to trigger fulfillment processes. This typically happens in real-time or near real-time as orders are placed.
* **What:** Approved sales orders from the ERP, containing all necessary details for the 3PL to pick, pack, and ship (e.g., order number, customer address, product SKUs, quantities, shipping method).
* **Direction:** **ERP ➔ 3PL**
* **Why:** ERP is the central hub. Once an order is validated and processed in the ERP, it triggers the fulfillment request to the 3PL. This ensures all financial and inventory adjustments are handled correctly before shipment.
* **What:** Confirmation that an order has been shipped (including tracking numbers, carrier details, shipped quantities), and any inventory adjustments (e.g., cycle counts, returns received at the warehouse, damaged goods).
* **Direction:** **3PL ➔ ERP**
* **Why:** This closes the loop. The ERP needs shipment confirmations to update order statuses, notify customers, and finalize financial records. Inventory adjustments from the 3PL are crucial for maintaining accurate stock levels in the ERP, which then propagates back to Shopify.
**Key Benefits of This First Integration**
* **Eliminates Manual Data Entry:** Drastically reduces errors and saves countless staff hours.
* **Real-time Inventory Accuracy:** Prevents overselling online and improves customer satisfaction.
* **Accelerated Order Fulfillment:** Orders move seamlessly from click to ship, improving delivery times.
* **Unified Data View:** Provides a single, accurate source of truth for sales, customers, and inventory across all key systems.
* **Scalability:** Lays the foundation for handling increased order volumes and future business growth without bottlenecks.
# No-Code UI Walkthrough
Source: https://docs.ekyam.ai/capabilities/no-code-configuration
This is a step-by-step walkthrough for a user to set up a data integration from NetSuite to Ekyam, culminating in a live data synchronization.
**Select the Connector**
Select the NetSuite Connector from the Ekyam Integration Options. Below is the screen that appears:
**Check Authentication**
On clicking the Netsuite connector, the user needs to fill the keys in the below screen to get the system connected to Ekyam Platform.
**Map Your Data (Universal Reader):**
***Click on Map Configurations***
After clicking on the Map Configurations, the screen below appears. The Products and Shipments data can be pulled using NetSuite's end-points.
The user needs to click on the Verify button, to come to the below screen.
***Note: The keys are mapped using AI, that validates the client’s keys.***
The data pulled from the Products and Shipments is then converted to Ekyam Data Standards.
However, the drop down mapping may differ for different systems.
**Configure Data flows**
***- Click on the Edit configuration***
After clicking on Edit configuration, below screen appears:
The above screen shows that the data has been configured. Then we see an "On sign" (or a green/active indicator) on a synchronization step like **"Shipment Sync"** in a workflow diagram, it precisely implies several crucial operational statuses:
1. **Active Connectivity:** It confirms that the connection between **Netsuite** (the source system for shipment data) and the Ekyam platform is live and operational. Data is actively flowing or ready to flow.
2. **Successful Configuration & Setup:** This indicator signals that all the necessary groundwork has been completed for this specific data flow. This includes:
* **Authentication:** Ekyam has successfully authenticated with Netsuite.
* **Field Mapping:** The critical step of **field mapping has been done**. This means Ekyam understands exactly which data fields from Netsuite's shipment records correspond to the fields required by the next system in the chain (Whiplash, in this case). **Workflow Logic:**.
3. **Data Synchronization is Live:** The "On sign" means that the **shipment products (or rather, the shipment data containing product information)** are actively being synced from Netsuite to the Ekyam platform, and then subsequently processed and delivered to the intended destination (Whiplash).
However, with the ***Ekyam’s Real-time Webhook/Event listening***, the data can be pulled in real-time. This robust framework ensures that when critical business events occur within Ekyam, external applications are notified and updated in near real-time, facilitating immediate response such as inventory synchronization or order fulfillment.
**Webhooks**
1. Developers register webhook endpoints for the events they want to receive.
2. Ekyam publishes events to Kafka topics.
3. A dedicated webhook consumer service reads from Kafka and delivers events to the registered webhooks.
4. *Robust Retry Logic:* Exponential backoff, maximum retry attempts, and a Dead-Letter Queue (DLQ) for failed deliveries.
5. *DLQ Monitoring:* Developers can view and manage messages in their DLQ via the Developer Portal.
6. *Alerting:* Notifications for DLQ events.
We do not have any writer mapping for Netsuite.
**Workflow of NetSuite**
This diagram illustrates a streamlined and automated **order fulfillment and inventory management process**. It connects the core business management (Netsuite) with the physical fulfillment (Whiplash) and the online sales channel (Shopify). Let's break it down:
**Core Systems Involved:**
1. **Netsuite:** This represents the **Enterprise Resource Planning (ERP) system**. Netsuite is a comprehensive cloud-based business management software that integrates financial management, ERP, CRM, and e-commerce functionalities.
2. **Whiplash:** This refers to **Whiplash Fulfillment**, which is a leading e-commerce fulfillment and warehouse management (WMS) provider. It handles the physical storage, picking, packing, and shipping of products for online retailers.
3. **Shopify:** This is a popular **e-commerce platform** where online stores are built and customer orders are placed.
**Data Flow Explained**
1. **Netsuite → Shipment Sync → Whiplash:**
This flow signifies that once an order is processed or marked for fulfillment within Netsuite (the ERP), the relevant shipment information is sent to Whiplash. This shipment data would typically include details necessary for the warehouse to fulfill the order, such as:
* Order details (items, quantities, customer address)
* Shipping method
* Customer information
* Any specific fulfillment instructions
This ensures that Whiplash receives all the necessary information to pick, pack, and ship the customer's order accurately and efficiently.
2. **Whiplash → Inventory Sync → Shopify**
As Whiplash fulfills orders (shipping items out) or receives new stock (inventory coming in), the actual, real-time inventory levels stored in Whiplash's WMS are updated. This crucial inventory data is then synchronized back to Shopify.
# Real-Time Integration
Source: https://docs.ekyam.ai/capabilities/real-time-integration
### **• Integration Patterns: Real-time vs. Batch**
This section will talk about how the users can choose an appropriate integration pattern based on the business needs and system capabilities.\
\
**Why is a Real-Time Integration needed?**
A Real-Time integration is needed as it allows to process and deliver the data as it gets created or processed. To understand why a real-time integration is necessary, let us highlight a few problems and their real-time solutions.
**→ To Prevent Costly Overselling and Stockouts:**
**Problem:** If the store’s inventory is not updated instantly when a product is sold or when a new stock arrives at the warehouse, there is a risk of overselling the product (Selling an item that is not available at the store).
**Real-time Solution**: Instant synchronisation of inventory across all channels (POS, WMS, e-commerce platforms etc.) to show accurate product availability to the customer.
→ **To enable quick fulfillment:**
**Problem**: Manual data entry or batch processing causes delays in the order-to-fulfillment cycle.
**Real-time solution**: Orders are pushed to fulfillment systems as soon as they are placed, thereby streamlining the supply chain.
### Webhooks: Enabling Real-time Integration
Based on the problems and solutions highlighted above, here’s how the webhooks help in Real-time Integration:
→ **Event-Driven Communication:** Webhooks are user-defined HTTP callbacks, which are triggered by specific events. Instead of calling an API to check on updates, a system can configure a webhook to be notified immediately when an event takes place.
When a product is sold (event), an e-commerce (POS) system can trigger a webhook. The webhook instantly sends data (Product ID, or inventory stock) to the inventory management system. The inventory system updates its stock at all channels. This process eliminates any delay that is associated with batch updates and reduces the risk of overselling.
When an order (event) is placed on an e-commerce platform, a webhook is fired. This webhook sends the order directly to the fulfillment system (WMS). This helps in beginning the fulfillment process quickly rather than waiting for a scheduled batch of orders.
**→ Push-based system:** Webhooks operate on a push-based model rather than a pull-based model. The push model ensures that the product is delivered as soon as it is available, which is fundamental to real-time integration.
# Brands
Source: https://docs.ekyam.ai/ekyam-data-standards/brands
This table stores the brand information.
# Business Entities
Source: https://docs.ekyam.ai/ekyam-data-standards/business-entities
It stores the business entity information.
[
](https://ekyam.ai/trial-form/)
# Businesses
Source: https://docs.ekyam.ai/ekyam-data-standards/businesses
It stores the business entity information.
[
](https://ekyam.ai/trial-form/)
# Carriers
Source: https://docs.ekyam.ai/ekyam-data-standards/carriers
It stores carrier information.
[
](https://ekyam.ai/trial-form/)
# Categories
Source: https://docs.ekyam.ai/ekyam-data-standards/categories
It stores category information.
[
](https://ekyam.ai/trial-form/)
# Customer Segments
Source: https://docs.ekyam.ai/ekyam-data-standards/customer-segments
It stores the customer information and profiles
[
](https://ekyam.ai/trial-form/)
# Customers
Source: https://docs.ekyam.ai/ekyam-data-standards/customers
This collection stores the customers information and profiles
# Inbound Receivings
Source: https://docs.ekyam.ai/ekyam-data-standards/inbound-receiving
It stores inbound receiving information.
[
](https://ekyam.ai/trial-form/)
# Inventories
Source: https://docs.ekyam.ai/ekyam-data-standards/inventories
This table stores the inventory information.
[
](https://ekyam.ai/trial-form/)
# Inventory Adjustments
Source: https://docs.ekyam.ai/ekyam-data-standards/inventory-adjustments
It stores the stock inventory and stock information.
[
](https://ekyam.ai/trial-form/)
# Locations
Source: https://docs.ekyam.ai/ekyam-data-standards/locations
It stores the information of the locations.
# Orders
Source: https://docs.ekyam.ai/ekyam-data-standards/orders
It stores the customers' order information.
[
](https://ekyam.ai/trial-form/)
# Partners
Source: https://docs.ekyam.ai/ekyam-data-standards/partners
It stores the partner and vendor information.
[
](https://ekyam.ai/trial-form/)
# Pricebooks
Source: https://docs.ekyam.ai/ekyam-data-standards/pricebooks
It stores pricing information.
[
](https://ekyam.ai/trial-form/)
# Prices
Source: https://docs.ekyam.ai/ekyam-data-standards/prices
It stores pricing information.
[
](https://ekyam.ai/trial-form/)
# Products
Source: https://docs.ekyam.ai/ekyam-data-standards/products
Stores the parent-level product information. It stores detailed product catalog data, including SKUs, descriptions, categorization, inventory, compliance, media, marketing, and variant details.
*The user can search different products by selecting the correct filters based on the chronicle model.*
[
](https://ekyam.ai/trial-form/)
# Promotions
Source: https://docs.ekyam.ai/ekyam-data-standards/promotions
It stores promotion and discount information.
[
](https://ekyam.ai/trial-form/)
# Purchase Orders (PO)
Source: https://docs.ekyam.ai/ekyam-data-standards/purchase-orders
It stores the customers' purchase orders information.
[
](https://ekyam.ai/trial-form/)
# Redemptions
Source: https://docs.ekyam.ai/ekyam-data-standards/redemptions
It shows the redemptions information.
[
](https://ekyam.ai/trial-form/)
# Refunds
Source: https://docs.ekyam.ai/ekyam-data-standards/refunds
The table stores the information of Refunds.
[
](https://ekyam.ai/trial-form/)
# Shipments
Source: https://docs.ekyam.ai/ekyam-data-standards/shipments
It stores the shipment and delivery information.
[
](https://ekyam.ai/trial-form/)
# Variants
Source: https://docs.ekyam.ai/ekyam-data-standards/variants
This collection models sellable item-level SKUs, scoped under products. Each variant contains detailed merchandising attributes, pricing, inventory summaries, channel-level configurations, media assets, and external integrations. It acts as the core transactional and fulfillment unit in most retail systems.
*The users will be able to search product variants by selecting the correct filter.*
[
](https://ekyam.ai/trial-form/)
# Vouchers
Source: https://docs.ekyam.ai/ekyam-data-standards/vouchers
It stores the voucher information.
[
](https://ekyam.ai/trial-form/)
# Glossary
Source: https://docs.ekyam.ai/glossary/glossary
Glossary
This glossary provides definitions for key terms, concepts, and technologies as they relate to the Ekyam Solution. Understanding these terms will help in better utilizing the platform's features and functionalities.
This glossary covers many of the key terms from our document. As we continue to expand our sections, more specific AI and developer-related terms can be added.
### **A**
* **Agentic AI**: The Ekyam solution layer utilizes AI, large language models (LLMs), and agent-based architectures to enable smart interactions, generate insights, and support autonomous operations for retailers.
* **AI Agents**: Specialized AI agents are designed to carry out tasks, support users, or make decisions within specific retail domains.
* **AI-Powered Data Mapping**: Ekyam uses Artificial Intelligence to analyze source and target data schemas (including EDI/iDOC structures) and suggest field mappings, simplifying integration setup.
* **Actions (Ekyam Workflows**): It includes specific actions performed within a workflow such as API calls, database lookups, EDI/iDOC processing, or sending notifications.
* **Apache Kafka**: An open-source, distributed event streaming platform used by Ekyam as its central message backbone for high-throughput, fault-tolerant, and scalable data handling.
* **Asynchronous AI Task Management:** The use of Apache Kafka within Ekyam to queue and manage AI tasks that are long-running, decoupling them from immediate user interaction.
* **API (Application Programming Interface)**: A set of rules and protocols that allows different software applications to communicate and exchange data. **Ekyam's Universal Connector** supports various API standards.
* **API Key Authentication**: An authentication method where an API key (a unique string) is used to grant access to an API. Supported by Ekyam's Universal Connector.
* **Authentication:** The process of verifying the identity of a user, system, or application attempting to access Ekyam. Ekyam primarily uses JWT (JSON Web token) for this.
* **Authorization:** The process of determining what actions an authenticated user, system, or application is permitted to perform within Ekyam, often managed by Role-Based Access Control (RBAC).
* **Authorization Code Grant (OAuth 2.0):** An OAuth 2.0 flow used by web applications where a user grants permission for an application to access their resources.
### **B**
* **B2B Document Exchange:** The process of businesses electronically exchanging structured documents like purchase orders, invoices, and shipping notices, often using EDI or iDOC formats. Ekyam facilitates this.
* **Batch Jobs:** A traditional integration method involving periodic extraction, transformation, and loading of data between systems at scheduled intervals. Ekyam aims to reduce reliance on these.
* **Basic Authentication (HTTP):** A simple authentication scheme built into the HTTP protocol, where the client sends a username and password with each request. Supported by Ekyam's Universal Connector.
### **C**
* **Chronicle**: Ekyam’s unified data engine that stores and contextualizes all product and inventory activity across the retail stack.
* **Canonical Data Models (Ekyam Data Standards):** Standardized representations for key business entities (e.g., Product, Order, Customer, Inventory) are defined by Ekyam to ensure consistent data interpretation.
* **CDC (Change Data Capture):** A technique for tracking and capturing data changes in a database by reading its transaction logs, enabling near real-time event detection for Ekyam Event Listeners.
* **Chunking (RAG):** The process of breaking down large documents into smaller, semantically coherent pieces for optimizing context retrieval for Ekyam's Retrieval-Augmented System (RAG).
* **Conditional Logic (Ekyam Workflows):** The ability within Ekyam Workflows to execute different branches of steps based on specified conditions (e.g., IF/THEN/ELSE, Switch/Case).
* **CRM (Customer Relationship Management):** Systems used to manage customer interactions, data, and relationships. Ekyam integrates with CRMs.
* **Configuration Isolation (Multi-Tenancy):** An architectural principle in Ekyam ensuring that each client’s specific configurations (e.g., connectors, workflows, EDI/iDOC profiles) are separate and do not affect other clients.
* **Confidence Scoring (AI Data Mapping):** A score provided by Ekyam's AI with each mapping suggestion, indicating the AI's certainty about the match.
### **D**
* **Data Isolation (Multi-Tenancy):** A core security principle in Ekyam ensuring that each client’s data is strictly separated and inaccessible to other clients.
* **Data Warehouses**: Ekyam integrates with the centralized repositories for storing large volumes of raw and processed data for analytics.
* **Data Mapping**: Ekyam does AI-powered data mapping. It is the process of defining correspondences between data fields from a source system and a target system.
* **Data Silos:** A situation where information is isolated within individual systems, preventing a holistic view of business operations. Ekyam aims to eliminate these.
* **Document Loaders (LangChain):** LangChain components used to ingest data from various sources (files, web pages) for processing in RAG pipelines.
* **Decoupling (EDA):** An architectural benefit of Event-Driven Architecture where event producers and consumers operate independently, communicating indirectly, which enhances resilience and scalability. Ekyam leverages this.
### **E**
* **eCommerce Platforms:** Eykam integrates with the software applications that power online stores.
* **EDA (Event-Driven Architecture):** It is an architectural paradigm where system behavior is orchestrated by the production, detection, and consumption of events. This is a core principle of Ekyam.
* **EDI (Electronic Data Interchange):** A set of standards for structuring information to be electronically exchanged between businesses. Ekyam provides robust support for parsing, processing and generating EDI documents.
* **Embedding Models (RAG):** Machine learning models used in Ekyam's RAG system to convert text or data chunks into numerical vector embeddings that capture semantic meaning.
* **ERP (Enterprise Resource Planning):** Integrated management software for core business processes. Ekyam integrates with ERPs, including SAP systems via iDOCs.
* **Event Listeners (Ekyam):** Components of Ekyam that actively monitor connected source systems (via polling, webhooks, CDC, file detection) for business events, acting as the platform's sensory network.
* **Event-Based Triggers (Ekyam Workflows):** Workflow initiators that automatically start a workflow when a specific event is detected by an Ekyam Event Listener.
### **F**
* **FTP/SFTP Servers:** File Transfer Protocol / Secure File Transfer Protocol servers used for file exchange. Ekyam's Universal Connector can connect to these, often for EDI/iDOC transfer.
* **File-Based Listeners (Ekyam):** Event listeners that monitor specific directories (e.g., on FTP/SFTP, cloud storage) for new or modified files, including EDI or iDOC documents.
* **Fine-Tuning (LLM):** The process of further training a pre-trained LLM on a smaller, domain-specific dataset to adapt its knowledge and improve its performance on particular tasks.
* **Functional Microservices (Ekyam):** Core Ekyam platform capabilities (e.g., authentication, order processing, inventory ledger) designed as independent, scalable microservices.
### **G**
* **Graphical Workflow Designer (Ekyam):** An intuitive, visual drag-and-drop interface within Ekyam for designing and configuring business process workflows.
### **H**
* **Hallucinations (LLM):** A phenomenon where LLMs generate plausible but incorrect or fabricated information. Ekyam's RAG architecture helps mitigate this.
### **I**
* **iDOC (Intermediate Document):** A standard data container format used by SAP systems for exchanging business transaction data (e.g., orders, deliveries, invoices) with other SAP or non-SAP systems.
* **iDOC Parsing Engine:** A component within Ekyam's Universal Reader responsible for interpreting the structure and content of incoming SAP iDOC files.
### **J**
* **JSON (JavaScript Object Notation):** A lightweight data-interchange format commonly used in REST APIs. Ekyam processes and generates JSON.
* **JWT (JSON Web Token):** An open standard for securely transmitting information between parties as a JSON object, used by Ekyam for authentication and authorization.
### **L**
* **LangChain:** An open-source framework used by Ekyam for developing applications powered by LLMs, providing modular components for chains, agents, memory, and tool usage.
* **LangGraph:** A library built on LangChain, used by Ekyam to create stateful, multi-actor AI applications with LLMs by modeling them as cyclical graphs.
* **LangSmith:** A platform for debugging, testing, evaluating, and monitoring LLM applications, used by Ekyam to ensure the observability and reliability of its LangChain-based AI agents.
* **Large Language Models (LLMs):** Advanced AI models (e.g., OpenAI, Gemini) capable of understanding and generating human-like text, forming a core part of Ekyam's Agentic AI.
* **Logical Data Segregation:** A primary method for data isolation using a client’s ID to ensure all operations are confined to the data of the authenticated client.
### **M**
* **Machine Learning (ML):** A field of AI that enables systems to learn from data without being explicitly programmed. Ekyam uses ML for AI-powered data mapping and potentially in its analytical models.
* **MCP (Model Context Protocol):** A conceptual framework inspiring Ekyam's approach to structuring how AI agents access and utilize context, tools, and resources for effective task performance.
* **Microservice Architecture:** Ekyam's platform-wide design where core functionalities and connectors are built as small, independent, and scalable services that communicate with each other.
* **Middleware:** Software that acts as a bridge between other applications, databases, and services, facilitating communication and data exchange. Ekyam functions as an advanced AI-powered middleware for retail.
* **Multi-Tenancy:** An architecture where a single instance of a software application (Ekyam) serves multiple clients (tenants) while keeping their data and configurations isolated and secure.
### **N**
* **Nodes:** Units of computation within a LangGraph, representing LLM calls, tool invocations, or custom functions, used to build Ekyam's AI agents.
* **Natural Language Processing (NLP):** A field of AI that enables computers to understand, interpret, and generate human language. Used by Ekyam's AI for conversational interfaces and data mapping.
* **NoSQL Databases:** Databases that do not use the traditional relational (tabular) model, such as document stores (MongoDB) or key-value stores (Redis). Ekyam can connect to these.
### **O**
* **OAuth 1.0/1.0a & OAuth 2.0:** Authorization frameworks that allow third-party applications to access user resources without exposing credentials. Supported by Ekyam's Universal Connector.
* **OMS (Order Management System):** Software that manages the entire order lifecycle. Ekyam integrates with OMSs.
### **P**
* **PIM (Product Information Management):** Systems for centralizing and managing product information. Ekyam integrates with PIMs.
* **Pinecone Vector DB:** A managed vector database used by Ekyam's RAG system for efficient semantic search and retrieval of contextual information to ground LLM responses.
* **POS (Point of Sale):** Systems used in physical stores to process transactions. Ekyam integrates with POS systems.
* **Polling (Event Listeners):** A method where Ekyam Event Listeners periodically query source systems to check for new or updated data.
* **Proactive Agents (Ekyam AI):** AI agents in Ekyam that can monitor events and data, identify situations requiring attention, and initiate actions or alerts without direct user command.
* **Prompts (MCP/LLM):** Carefully engineered instructions given to LLMs to guide their behavior, reasoning, and response generation within Ekyam's AI.
* **Protocol Handling (Universal Writer):** The capability of Ekyam's Universal Writer to use the appropriate communication protocol (e.g., HTTP, SFTP) to deliver data to destination systems.
### **Q**
* **Quantity Committed:** Inventory reserved for sales orders but not yet shipped. Tracked by Ekyam Universal Ledger.
* **Quantity OnHand (QOH):** Total physical stock at a location. Tracked by Ekyam Universal Ledger.
* **Quantity OnOrder:** Stock ordered from suppliers but not yet received. Tracked by Ekyam Universal Ledger.
### **R**
* **RAG (Retrieval Augmented Generation):** An AI architecture used by Ekyam that enhances LLM responses by first retrieving relevant factual context from external knowledge sources (like Pinecone) and providing it to the LLM.
* **RBAC (Role-Based Access Control):** A security model used by Ekyam to manage user permissions based on assigned roles within a tenancy.
* **Redis:** An in-memory data store used by Ekyam for high-performance caching in its AI systems, reducing latency and costs.
* **Refresh Token Grant (OAuth 2.0):** An OAuth 2.0 mechanism to obtain new access tokens without requiring user re-authentication.
* **Resilience (Architecture):** The ability of the Ekyam platform and its microservices to withstand and recover from failures, ensuring high availability.
* **Resources (MCP):** Data and knowledge assets (e.g., Universal Ledger data, EDI/iDOC content, product catalogs) that Ekyam AI agents can access and reason over.
* **REST (Representational State Transfer) APIs:** A common architectural style for web APIs, widely supported by Ekyam's Universal Connector.
* **Retry Mechanisms (Kafka/Workflows):** Ekyam's capability to automatically retry failed operations (e.g., API calls, event processing) a configured number of times to handle transient issues.
* **RouterChain (LangChain):** A LangChain component used by Ekyam's intelligent LLM router to decide which LLM or prompt to use for a given query.
### **S**
* **Safety Stock:** A buffer inventory level maintained to prevent stockouts. Considered by Ekyam's Universal Ledger.
* **Scalability (Architecture):** The ability of the Ekyam platform and its components (like Kafka and microservices) to handle increasing volumes of data and transactions by adding resources.
* **Schema Analysis (AI Data Mapping):** The process by which Ekyam's AI ingests and understands the structure of source and target data to suggest mappings.
* **SCM (Supply Chain Management) Systems:** Software used to manage the end-to-end flow of goods, information, and finances in a supply chain.
* **SDKs (Software Development Kits - Ekyam):** Libraries and tools provided by Ekyam (initially in Python) to enable developers to build custom extensions and integrations for the platform.
* **Semantic Search (RAG/Pinecone):** Searching for information based on meaning and context rather than just keywords, enabled by vector embeddings and Pinecone in Ekyam's RAG.
* **SequentialChain (LangChain):** A LangChain component for linking multiple chains or calls in a sequence, where the output of one becomes the input to the next.
* **Single Source of Truth (SSoT):** A central, reliable, and consistent repository of data (like Ekyam's Universal Ledger) that all connected systems can trust and use for decision-making.
* **System Prompts (MCP/LLM):** High-level instructions defining an Ekyam AI agent's persona, objectives, and constraints.
### **T**
* **Tenant ID (Multi-Tenancy):** A unique identifier used in Ekyam to associate all data and configurations with a specific client, ensuring logical data segregation.
* **Text Splitters (LangChain):** LangChain components used to break down large documents into smaller chunks for effective processing in RAG pipelines.
* **Tools (MCP/LangChain):** Capabilities or actions (e.g., API calls, database queries, analytical model invocations) that Ekyam AI agents can perform to interact with their environment.
* **Triggers (Ekyam Workflows):** Mechanisms (event-based, scheduled, manual, API) that initiate the execution of an Ekyam Workflow.
### **U**
* **Universal Connector :** A versatile Ekyam component, built as microservices, that enables connections to a wide array of external systems, APIs, and data sources using various protocols and authentication methods.
* **Universal Ledger:** A centralized, real-time, standardized data store within Ekyam that acts as the definitive source of truth for key operational data like inventory, orders, and customer information.
* **Universal Reader:** An Ekyam component responsible for ingesting raw data from various source systems (including parsing EDI/iDOCs) and transforming it into Ekyam Data Standards.
* **Universal Writer:** An Ekyam component that takes standardized data from within Ekyam and transforms/formats it for delivery to destination systems or trading partners, including generating EDI/iDOC documents.
### **V**
* **Vector Embeddings (RAG):** Dense numerical representations of text or data that capture semantic meaning, used by Ekyam's RAG system with Pinecone for similarity searches.
* **Vector Store (Pinecone):** A specialized database, like Pinecone, used by Ekyam to store and efficiently query vector embeddings for its RAG system.
### **W**
* **Webhooks (HTTP Callbacks):** A method where a source system actively sends an HTTP notification to an Ekyam endpoint when an event occurs, enabling real-time data capture.
* **WMS (Warehouse Management System):** Software for controlling and optimizing warehouse operations. Ekyam integrates with WMSs.
* **Workflows (Ekyam):** The Ekyam component for defining, orchestrating, and automating multi-step business processes that span across integrated systems, incorporating transformations, business logic, and AI capabilities.
### **X**
* **XML (Extensible Markup Language):** A markup language for encoding documents in a format that is both human-readable and machine-readable. Handled by Ekyam's Universal Connector, Reader, and Writer.
### **Z**
**Zep:** A platform for persistent, long-term memory for LLM applications, used by Ekyam to enhance the conversational coherence and context-awareness of its AI agents.
# Anatomy of Operational Intelligence Hub
Source: https://docs.ekyam.ai/intelligence-hub/operational-intelligence-hub
A Visionary Guide to Operational Intelligence Hub
The Operational Intelligence Hub is the **fourth layer of Ekyam’s Platform Architecture**. It is the primary interface where the business users interact with the insights and power of the Ekyam Platform. More than just being a reporting tool, it is a unified, dynamic system where the users can monitor, analyze, and transform the complex, real-time data from the Ekyam Retail Knowledge Graph (RKG) into an intelligent action.
The hub is the COMMAND Centre, that allows the user to:
→ **Monitor:** Carefully monitoring the Key Performance Indicators (KPIs), and receiving alerts for critical events such as low stock levels etc.
→ **Analyze:** Identify the issues and work towards implementing a solution.
→ **Act:** Automate responses and trigger workflows directly from the insights.
### **• Ekyam Reports: Help to Understand the Past Performance**
Ekyam reports will help the users to understand trends, evaluate past performances, strategize plans by selecting the metrics and dimensions directly from the ontology.
**A Use Case Unveiled:**
Let us understand a common use case when a retailer is trying to analyze the performance of a recent collection. There could be a possibility if a “Quarterly Product Performance Report” could be generated that presents itself with the following structure:
→ **Goal**: To understand which products in the "Spring 2025 Collection" were most and least successful.
→ **Filters**: Collection.collection\_name = "Spring 2025", Timeframe = "Q1 2025".
→ **Dimensions** (Rows): The report would be structured by Product.product\_name and broken down further by each Item.sku.
→ **Metrics** (Columns): Key metrics would include Net Revenue, Units Sold, Gross Profit Margin, Sell-Through %, and Return Rate..
*This report may provide the user a defined, SKU- level breakdown insight for further performance analysis.*
### **• Dashboards for Monitoring Performance**
Dashboards are designed to gauge performance, monitor the health of the business and investigate issues like a sudden increase in the price of a specific product. The dashboards are composed of multiple widgets like graphs, charts, that will help the user to real-time track the **Key Performance Indicators** (KPIs). It is like giving a dynamic and a live-view to the current operations to make the user understand the immediate need of the business requirements.
| **Core Components of a Dashboard**
1. **Widgets:** Displays information in the form of Graphs, Charts or Tables and provide a quick overview of a KPI
2. **Key Performance Indicators (KPIs):** Measurable metrics used to track the progress.
3. **Data Source:** A system or a database from which the dashboard pulls its information |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
**Let us understand a Use case:**
An e-commerce Manager would rely on “Daily e-commerce Operations Dashboard” to understand the entire activity. The dashboard in this case might include:
* A **“Today’s Net Revenue’’** scorecard that provides a quick snapshot of a financial KPI and a Year-On-Year (YOY) comparison to understand the performance.
* An **“Orders By Hour”** **Line chart**, which is a strong visualization tool to indicate trends and patterns in sales.
* The **“Top 5 Returns Reasons” chart** gives an immediate insight into customer feedback by showing the most common Return.return\_reason codes like "Wrong Size" or "Damaged".
* The “Live BOPIS Orders” Map widget, shows which Locationentities are handling a "Buy Online, Pick-up in Store" orders, to help monitor fulfillment efficiency.
*This Use case illustrates how a well-designed dashboard, with its specific combination of widgets, KPIs, and real-time data,helps to not only monitor performance but also to make quick, data-driven decisions.*
### **• Alerts: Proactive Warnings for Critical Events**
Position alerts are the automated early-warning system of Ekyam’s data platform. These alerts are the proactive notifications that are triggered when a specific, critical condition or a pre-defined schedule is met within the RKG. The Alerts are vital for users to prevent potential problems before they escalate, thereby reducing significant damage.
**Use Case**
**Low weeks of Supply (WOS) Alert**
A crucial use case to understand:
→ **It Monitors:** All Item entities are in “Active”Selling status.
→ **Its Condition:** The alert triggers the moment the calculated Weeks of Supply (WOS) for any of these items drops below a set value, say 3 weeks. This threshold is dependent on customized factors like lead time for new orders or popularity of the product.
→ **The Logic:** The system continuously evaluates the Inventory\_Level.sellable\_qty in real-time from the RKG to show an accurate measure of how quickly the product is selling.
→ **The Result:** The system immediately sends an alert to the team, that allows them to proactively initiate a new PurchaseOrder before the item goes out of stock.
### **• Notifications: Informational Updates**
Notifications are automated informational updates that keep the users aware of normal and ongoing business processes. They are designed to keep the team informed to ensure that all the stakeholders are aware that a key action has been completed and the process can move on to the next stage.
**Use case:**\
Let us see an example of **“Purchase Order Received”** notification.
→ **It Monitors**: The PurchaseOrders Quantity
→ **Its Condition:** A notification is sent when a PurchaseOrders.status changes from “In-transit” to “Received”.
→ **The Trigger:** The event is triggered when the warehouse staff scans the items from the PurchaseOrder shipment,the system updates the Inventoriesl.on\_hand\_qty in the RKG.
→ **The Result:** A notification is sent to the inventory manager, who understands that the stock is available for allocation; and to the finance department to make them aware that the invoice from the Partners can be approved for payment. This automation reduces manual effort, streamlines the workflow and ensures that the teams are working with most updated information.
### **• Workflows: Automating Business Logic**
Workflows are the most powerful capability of the Operational Intelligence Hub. It is an automated, multi-step series of actions that are triggered by an alert. This capability is not simply to notify the users, but to actively perform tasks, orchestrate processes and connect Ekyam to other applications in the technology stack.
Let us understand the below Illustration: **High-Value Customer Retention Workflow**
**→ The Trigger:** The workflow is initiated by a critical event. An Alert is configured to fire when a Customers who is flagged as a "VIP" initiates a Returns for an Orders where the net\_revenue was greater than \$500. This highly specific condition ensures that the workflow is only activated for the most valuable customer interactions.
→ **Workflow Steps:**
* **Step 1 (Internal Tagging):** The system automatically finds the corresponding Returns record and applies a "VIP Priority" tag, ensuring it is handled first.
* **Step 2 (Team Notification):** The workflow sends a high-priority message to the VIP customer internal communication channel, containing the customer\_id and order\_id, which helps the team to reach out to the customer if necessary.
* **Step 3 (Coupon Generation):** The workflow internally generates a unique, single-use Vouchers for the customer's next purchase.
* **Step 4 (Personalized Outreach):** The workflow executes an API call to the company's email marketing platform to send a personalized email to the VIP customer. The email uses a template like "We've processed your return.." and includes the newly generated voucher code as a "thank you" for their loyalty.
Ekyam’s workflow automates complex business processes by linking data, action and logic. This eliminates the manual effort by orchestrating multi-step responses to specific events.
# Configuration Manual
Source: https://docs.ekyam.ai/introduction/configuration-guide
### **• Setup Guide**
This section will guide you through the typical steps involved in establishing a connection between a Universal Connector and the Target System. Whether it is to synchronize customer data, automate order fulfillment, or enable real-time inventory updates, understanding the steps/process is crucial for leveraging the Ekyam integration platform. An outline of the general workflow—-from initial setup and authentication to data mapping and testing is explained below:
### **Step 1: Initial Setup & Configuration (Ekyam & External System)**
Some systems (like those using OAuth 2.0) require Ekyam to register. This involves obtaining credentials like a Client ID and Client Secret.
Determine the authentication method used by the system.The common types include:
* **OAuth 2.0:** Requires a redirect flow and the exchange of authorization codes for access and refresh tokens.
* **API Keys:** Simple string tokens passed in headers or query parameters.
* **Basic Authentication:** Username and password
* **Bearer Tokens (JWT):** JSON Web Tokens, often used in API authorization.
Based on the authentication type, obtain the required credentials.
* **OAuth 2.0**: Client ID, Client Secret, Redirect URIs (obtained during app registration).
* **API Keys**: Retrieve the API key from the target system's developer portal or settings.
* **Basic Authentication**: Obtain the username and password
* **Bearer Tokens (JWT)**: Obtain the token from the target system, potentially through a login API call.
### **Step 2: Connection Setup with Ekyam**
On navigating to the Ekyam Integration options, there will be an ADD/SELECT CONNECTOR button to the list of available connectors. Select the SOURCE SYSTEM CONNECTOR for integration (NetSuite, SAPB1 etc).
Once the desired connector has been selected, the system will prompt for authentication details. This is a vital step as it establishes a secure and authorized connection between the Source System and the Ekyam Platform.
On the screen that appears, there will be fields to input the Source System's credentials. These might include API Keys, Client IDs, Client Secrets, Usernames, Passwords, or initiating an OAuth 2.0 redirect flow depending on the Source System's security protocol. These unique credentials allow Ekyam to securely connect and access your data.
### **Step 3: Configuration and Testing**
**Map Your Data (Universal Reader)**
Once Ekyam has successfully authenticated with the Source System, it is important to define how the external data will be understood and standardized by Ekyam’s Proprietary schema. Hence, the role of “UNIVERSAL READER”.
Begin by clicking on the "Map Configurations" button or tab.
After clicking on "Map Configurations," the system prepares for data ingestion. A mapping screen will appear (Shown below), indicating the types of data that can be pulled from your Source System's designated API endpoints (e.g., /products, /customers, /orders).
To confirm the validity of these endpoints and the established connection, you will need to click on the "Verify" button.
**“AI-Powered Mapping”:** Ekyam's Universal Reader leverages advanced AI-driven capabilities during this process. The AI intelligently validates the connection and begins to infer and suggest preliminary mappings based on common field patterns and industry best practices. This significantly streamlines the setup and reduces manual effort.\_
**Successful Verification & Reader Mapping Screen**\
If the "Verify" check is successful, the Reader Mapping Screen will be displayed. This intuitive interface is where the transformation of the Source System data into Ekyam's unified standards can be seen.
* **Ekyam’s Standardized Collections (Left Panel): O**n the left side of the screen, you'll see Ekyam's pre-defined, standardized data collections and their respective keys/fields. These represent the target schema for your data—Ekyam's proprietary data standards—structured to store core business entities (e.g., Products, Pricing, Inventory, Customers, Orders) and maintain their relational integrity within Ekyam's ecosystem. These are the standardized keys your data will conform to.
* **Source System Fields (Right Panel):** Conversely, the right side of the screen dynamically displays the actual fields/keys retrieved directly from your connected Source System's API endpoints. This clear juxtaposition allows you to visually connect the source data elements with Ekyam's standardized fields.
As you select a specific Ekyam collection (e.g., "Products") on the left, the main screen area will display the detailed field mapping. Ekyam’s standard keys for that collection (e.g., Product ID, Product SKU, Product Name) are shown. As data is received from the Source System, Ekyam's AI intelligently maps the incoming data from the right panel to these defined Ekyam Data Standards on the left.
* **Custom Mappings (Optional):** While AI suggests mappings, the dropdown options allow for manual adjustments. If certain fields from the Source System are not automatically mapped, or if you prefer a different mapping, you can define Custom Mappings manually. These custom rules are saved in Ekyam's database for consistent application during subsequent syncs, providing unparalleled flexibility.
***The specific dropdown mapping options may differ significantly for different Source Systems due to their unique data structures.***
### **Step 4: Configure Data Flows (Workflows)**
With your data mappings defined, the final step is to activate and control the flow of information through Ekyam's powerful workflow engine.
* Click on the Workflow to edit the data flow
* After clicking, the screen will display your configured data flows.
* You will observe an "On sign" (often a blue toggle or an active indicator) on a specific synchronization step within your workflow diagram, such as "Order Sync" or "Shipment Sync."
This "On sign" precisely implies several crucial operational statuses for a seamless data exchange:
* **Active Connectivity:** It confirms that the connection between your Source System (e.g., for shipment data) and the Ekyam platform is live and operational. Data is actively flowing or ready to flow.
* **Successful Configuration & Setup:** This indicator signals that all the necessary groundwork has been completed for this specific data flow. This includes:
* **Authentication:** Ekyam has successfully authenticated with your Source System.
* **Field Mapping:** The critical step of field mapping has been precisely done, meaning Ekyam understands exactly which data fields from your Source System's records correspond to the fields required by Ekyam's standards and, subsequently, any other destination system in the chain.
* **Workflow Logic:** Any defined business rules, transformations, or AI-powered actions (as configured in Ekyam Workflows) are active and ready to be applied.
* **Data Synchronization is Live:** Most importantly, the "On sign" signifies that the data (e.g., product updates, customer records, or shipment details) is actively being synced from your Source System to the Ekyam platform, processed according to your defined mappings and workflows, and then subsequently delivered to its intended destination (e.g., your WMS, CRM, or data warehouse).
**Use Case**
To explain how Ekyam Universal Connector is using Diverse API standards, Below is a **Use Case of NetSuite** that will demonstrate how Ekyam's Universal Connector leverages NetSuite's diverse API standards (APIs, Token-Based Authentication (TBA), Webhooks) and authentication mechanisms (token-based, API key) to achieve seamless, real-time order-to-fulfillment synchronization.
**Scenario:** An e-commerce uses Order Management System (OMS), and Ekyam’s Universal Connector to integrate with NetSuite (ERP) for inventory and fulfillment management.
\
***Ekyam's Universal Connector needs to connect to NetSuite for several real-time operations, leveraging NetSuite's various API standards and authentication methods.***
* **Connecting to NetSuite (Authentication & Token Generation)**
NetSuite offers several API methods, including:
\
→ Token-Based Authentication (TBA)- based on OAuth 1.0
→ Oauth 2.0
* **Ekyam’s Configuration for Order Creation (OMS to NetSuite)**
* The user inputs Consumer Key, Consumer Secret, Access Token, Token Secret to connect Ekyam’s Universal connector to Netsuite.
* The Universal Connector then uses these credentials to generate an authenticated request with OAuth 1.0 (for TBA) or OAuth 2.0 token (Client credentials).
* This token is then passed as a header in every API call to Netsuite.
* The token acts as a proof of authentication and authorization.
* **Using webhooks for Inventory updates**
* NetSuite is configured to trigger an event whenever an inventory’s level changes.
* Ekyam’s Universal Connector provides a webhook endpoint URL to NetSuite.
* Ekyam generates a unique callback URL for NetSuite to include it in its webbook payload or header when sending inventory updates.
* When an inventory event occurs in NetSuite, it **\_pushes \_**\_a JSON payload \_containing the updated inventory data to Ekyam’s webhook endpoint, with the API key included for authentication.
* **Data flow**
* When a customer places an order in the OMS, an event is triggered.
* Ekyam's Universal connector, authenticated with NetSuite's TBA-generated token, immediately calls NetSuite's API to create a new sales order record.
* Ekyam maps OMS order fields according to Ekyam Data Standards to NetSuite's sales order fields.
* **Inventory Synchronization**
* When an item is sold, received, or adjusted in NetSuite, the inventory record changes.
* NetSuite triggers a webhook that *pushes* the updated inventory quantity for the item to Ekyam's webhook endpoint.
* Ekyam maps NetSuite's **quantity available** and **item\_id** to the OMS's equivalent fields.
* **Fulfillment Updates**
* Once a sales order is fulfilled in NetSuite, an event is triggered.
* NetSuite pushes a webhook to Ekyam's endpoint, providing fulfillment details
* Ekyam's connector receives this, maps the data, and updates the order status in the OMS.
# Connection Protocols
Source: https://docs.ekyam.ai/introduction/connection-methods
### **Connection Methods**
A core component of Ekyam's versatility is its “**Universal Connector”**. This powerful microservice-based connector is designed to establish connections with a vast array of systems and data sources commonly found in the retail ecosystem, "with ease." Ekyam’s Universal Connector supports a broad range of data exchange methods. Furthermore, we have Universal Authentication support to securely access any system.
The **Universal Connector** supports Ekyam’s data exchange methods:
* **REST APIs:** They are secure and scalable connections via HTTP/HTTPS, supporting JSON and XML payloads. APIs are needed for real-time data exchange with modern Saas and Web services. The choice of authentication methods depends on the specific system being integrated. These APIs are widely used for modern web services.
* **API key**: Ekyam uses a unique key that is issued by one system to another. The client system includes this key in its request to the server API. In addition, the API Secret key varies from system to system.
* **OAuth 1.0 & OAuth 2.0** (Authorization Framework for Delegated Access): Ekyam uses OAuth 2.0 JWT **authorization framework** that allows a user to grant the client **limited access** to their resources on a service without sharing their login credentials with the third-party app. It does this by issuing access tokens. Implying that these are secure authorization protocols for API access.
**Secure Login:** *Access to the Ekyam platform is protected by strong authentication mechanisms, ensuring only authorized personnel can access and configure integrations.*
* **System Authentication to Hit the API**: While OAuth 2.0 enables a client system to hit an API, it is doing so on behalf of a user or on its own behalf (using client credentials).
**If the system uses: Oauth1.0:** The system that uses Oauth1.0 needs to fill in: Signature Method, Consumer Key, Consumer Secret, Access Token and Token Secret, after that their system gets connected. :
* **Basic Authentication:** Basic Authentication refers to the use of a **standard username and password combination** to verify identity. When an application or user attempts to connect, these credentials are sent, and if they match the records, access is granted.
* **Digest Authentication:** It is a more secure challenge-response authentication method. Unlike Basic Authentication, which transmits credentials in an easily decodable (Base64 encoded) format, **Digest Authentication is a more secure challenge-response authentication method.** It's designed to ensure that a user or system proves their knowledge of a password without ever sending the password itself in plain text across the network.
### • Various Data Sources
* **Database Connectivity:** Whether it's the transactional history from the POS system or the intricate product details managed in a specialized internal database, Ekyam provides the capability for direct access to the enterprise data stores and systems of record.
Ekyam’s Universal connector **supports DBs:** PostgreSQL, MySQL, MongoDB. In addition to this, Ekyam reads and writes data to these DBs.
* **Structured Documents:** For handling complex B2B and enterprise formats like EDI and iDOCs, Ekyam has an extensive EDI module that can read EDI-standard data from various systems, transform it, and map it to other systems for seamless integration across platforms.
* **Cloud File Storage:** Ekyam ensures that retailers can leverage the immense power and scalability of modern cloud infrastructure by seamlessly integrating with leading **Cloud File Storage** services. Ekyam deeply integrates with the industry's top-tier cloud storage providers like Amazon S3, Google Cloud Storage, Azure Blob Storage.
* **Data Lakes:** Ekyam's robust capabilities ensure it **facilitates seamless connection to the centralized data repositories**, be it structured (like database tables), semi-structured (like XML or JSON files), or unstructured (like images, audio, or social media posts), serving as the crucial bridge between your operational systems and your comprehensive data strategy.
* **File-Based Workflows (Cloud & Local):** Ekyam’s Universal Connector can even leverage **email attachments (via services like Gmail)** as powerful triggers for automated workflows.
**Use case: The "Email Parsing Agent" (FastAPI Endpoint)**
1. Imagine your 3PL (Third-Party Logistics) partner sends you a daily email with **a sales\_report.csv** file attached.
2. Ekyam deploys a specialized component, which we can call the "Email Parsing Agent." This agent is exposed as an endpoint within Ekyam's FastAPI integration layer.
3. When a new email matching specific criteria (e.g., sender, subject line) arrives in your connected inbox (e.g., Gmail), a mechanism (it could be a webhook from Gmail) **"hits" this FastAPI endpoint on the Ekyam server.** This "hit" acts as the trigger for the workflow.
There are multiple AI agents that are working:
**AI Agent 1: Automated Data Retrieval**
* Once the FastAPI endpoint is hit, a predefined function is executed on the Ekyam server.
* This function's immediate task is to go into that specific email body, identify, and securely fetch the attached CSV file.
* This initial data retrieval from the external source (the email/3PL) is handled by the first of Ekyam's multiple AI agents, specializing in data retrieval. It ensures the data is safely pulled from its source and brought into Ekyam's processing environment.
**AI Agent 2: The Parsing Agent**
* Now that the raw CSV file is within Ekyam's system, it's passed to the second AI agent: the "Parsing Agent.
* This AI agent doesn't just read the file; it intelligently parses its content. It understands the structure of the CSV (columns, data types), identifies key metrics (e.g., product IDs, quantities sold, revenue, dates), and even cleans or normalizes the data.
**AI Agent 3: The Data Sender**
* Finally, the clean, parsed, and often enriched data is handed over to the third AI agent.
* This agent's role is to send the processed response to its designated place. This could mean:
* Pushing the sales data into your ERP (e.g., Netsuite or SAP).
* Updating inventory levels in your WMS or e-commerce platform (e.g., Shopify).
* Feeding a data lake for analytical processing.
* Triggering another automated workflow
# Data Workflows
Source: https://docs.ekyam.ai/introduction/data-workflows
### **• Orchestrating Data Workflows**
An **Ekyam Workflow** is a configurable, automated sequence of data transmissions and operations that defines how Ekyam processes events and moves data between one or more source systems and one or more destination systems. Workflows are the core of Ekyam's business process automation capabilities.
**Defining Business Logic**
Ekyam Workflows allow users to embed sophisticated business rules and logic directly into the integration flows without extensive custom coding. This is achieved through a user-friendly interface that supports various operations, including:
* **Mathematical Operations:** Perform calculations on data fields (e.g., calculate total order value including tax, convert currencies).
* **Logical Operations:** Implement conditional logic (IF-THEN-ELSE statements) to route data differently based on specific criteria (e.g., if order value > \$500, flag for expedited shipping).
* **String Operations:** Manipulate text data (e.g., concatenate address fields, parse product descriptions, change case).
* **Database Operations:** Look up additional information from connected databases to enrich data in transit (e.g., fetch customer loyalty status based on customer ID).
* **Transformations:** Apply complex data transformations beyond simple field mapping.
### **• AI-Powered Workflow Capabilities**
Ekyam Workflows integrate advanced AI capabilities, empowering users to build even more intelligent automations:
* **AI-Based Queries:** Users can employ natural language prompts to query data within the Ekyam ecosystem or connected systems. Eg. RDBMS
* **AI-Generated Code/Logic:** For more complex or custom requirements, users can describe the desired logic in natural language. Ekyam's AI can then assist in generating the underlying code or workflow steps needed to implement that logic. The user can review and refine this AI-generated logic. Eg. Custom operations
* **Predictive Analytics Integration:** Workflows can trigger or be triggered by AI models, for example, to predict fraudulent orders, forecast demand based on incoming sales data, or personalize product recommendations.
Let us explain the above using a Use Case of where the NetSuite data needs to be pushed to Shopify (NetSuite Shopify Products Sync):
**Source**: Net Suite ; **Destination**: Shopify
To show the different types of Sync in this Data flow, the user can click: Order Sync, Inventory Sync, Products Sync , Shipment Sync and Data Sync.
For applying Business Logic, the user can choose different fields:
***A user can choose multiple functions to create a workflow.***
**Workflow Representation:**
The below screenshot shows the Products Sync between NetSuite and Shopify.
**String Operations**
When a user selects String Operation, below is the screen that will appear.
**Mathematical Functions**
On selecting the Mathematical functions, below screen is visible:
**Date Operations**
The user can select the Date Operations as shown below:
**RDBMS functions**
On selecting the RDBMS Operations: The user will be able to see a User Prompt and Aggregation text box. The user can write the prompt in it to generate an Aggregation.
**Custom Operations**
When the user selects Custom Operations, they can add a User prompt to generate a Code snippet.
# Monitoring Dashboard
Source: https://docs.ekyam.ai/introduction/monitoring-dashboard
### **• Monitor & Troubleshoot Connectors**
The Monitoring Dashboard provides real-time insights into data-flow, allowing you to quickly identify and address issues.
### **• Key Metrics and Interpretation**
The dashboard presents various key metrics that are essential for understanding performance of Ekyam’s data connectors.
**Successful Transactions:** The successful transaction metric indicates the number of records or data that have successfully completed their journey through the connectors and workflows.
**Pending Tasks (Queue Tracker):** The Ekyam’s Queue Tracker shows the number of data items currently awaiting processing in various queues.
**Errors/Failed Transactions:** This critical metric highlights the number of transactions that encountered an error during processing.
# Introduction to Ekyam
Source: https://docs.ekyam.ai/introduction/overview
### **• Ekyam Platform Overview**
Ekyam Platform is built upon a proprietary data platform engineered to transform raw data into an AI-ready format. At its core, it provides a comprehensive library of AI agents, each specialized in retail operations, designed for seamless collaboration amongst themselves and with other 3P AI agents.
Ekyam Platform enables real-time communication and data synchronization across all connected retail systems, regardless of their underlying technology or data structure, offering a solution with its AI-powered retail integration platform designed to make your data work smarter.
### **• Platform Architecture**
### Ekyam Universal Connector
It is a versatile Ekyam component, built as microservices, that supports all common protocols and formats (APIs, FTP/SFTP, databases, XML/JSON files, EDI X12/EDIFACT, etc.) with no-code configuration. Each connector ingests or sends data in any format and automatically to Ekyam’s canonical schema. The external systems (ERP, OMS, WMS, POS, ecommerce, etc.) are connected via Ekyam’s pre-built connectors, which extract data (orders, shipments, inventory updates, vendor info, etc.) in real time. These connectors feed data into the core platform. Ekyam’s integration layer “serves as a bridge” between disparate retail systems, providing a unified, real-time view of inventory and orders.
**Key Features**
* The connector layer is extensible – custom connectors and mappings can be added – and fully monitored (tracking throughput, errors, SLAs). For instance, connecting an ERP’s flat-file export takes minutes by configuring endpoints; the system then uses AI-driven profiling to map fields and validate formats.
* The connector also handles bi-directional flows: outbound documents are generated from the Chronicle data and sent to partners. All flows are logged and audited to ensure visibility.
* With built-in **monitoring, throttling, retry logic**, and **anomaly detection**, the Universal Connector ensures data integrity, availability, and security across the entire retail stack.
**Security Highlights:**
* ***End-to-end encryption*** for all data in transit and at rest (TLS/SSL, AES-256)
* ***Token-based and certificate-based authentication***
* ***Role-based access control (RBAC)*** and data-level permissions
* ***Audit trails and activity logging*** for every transaction and event
* ***Compliance-ready*** architecture aligned with SOC 2, GDPR, and ISO 27001 best practices.
### Ekyam Universal Reader
An Ekyam component responsible for ingesting raw data from various systems, including parsing EDI/iDOCs and transforming it into Ekyam Data Standards.
**Key Features**
* It has the intelligence to process an old EDI file, a new API feed or a PDF invoice.
* This component ensures that Ekyam’s AI-native platform can seamlessly integrate with and understand the entire spectrum of a retailer’s data ecosystem.
* It has the capability of converting all incoming data into an understandable format that can be used to build the Retail Knowledge Graph (RKG) and AI agents. This processed data is then available for RKG to build connections and for AI agents to make intelligent decisions and automate workflows.
### Ekyam Universal Writer
It supports CSV, XML, JSON, TXT, etc., for internal systems, reports, or custom integrations.
**Key Features**
* The "Universal Writer" ensures that data is sent automatically based on business rules or AI-driven triggers (e.g., automatically sending a shipping notification once an order is fulfilled or updating stock levels on all sales channels simultaneously).
* It ensures that the valuable insights and automated actions generated by Ekyam's AI and Retail Knowledge Graph are not confined to the platform but can flow seamlessly throughout the entire retail ecosystem, enabling true operational efficiency and real-time decision-making.
### Ekyam Universal Ledger
A centralized, real-time, standardized data store within Ekyam that acts as the definitive **Source of Truth** for key operational data like inventory, orders, and customer information.
**Key Features**
* The **Ekyam Universal Ledger** is the platform's core data backbone, creating a **unified, real-time record** of all critical retail operations.
* It consolidates retail data into a single, intelligent, and continuously updated record, empowering the entire Ekyam platform to deliver true AI-driven efficiency and visibility for retailers.
### Retail Knowledge Graph (RKG) Core
Ekyam’s Retail Knowledge Graph (RKG) organizes all core retail data, products, SKUs, categories, vendors, inventory, orders and store locations into a unified, intelligent model. It is a graph database that models retail entities (products, SKUs, categories, vendors, locations, customers, etc.) and their relationships. The graph encodes the semantic schema (ontology) of the retail domain (for example, “Product A is supplied by Vendor X” or “SKU123 is stocked at Warehouse Y”). This structured representation provides a “system of truth” for facts about products, vendors, and static reference data. Natural language queries about product details or vendor relationships are grounded in the RKG, which can be queried via graph query APIs or natural-language-to-graph translations.
**Key Features**
* The RKG will make it easier to answer business-critical queries like: "Which products in size XL sold at Store 123 last month?” or “Which vendors frequently short-ship a specific SKU?”
### AI-Native Middleware
The top layer consists of intelligent agents and AI-driven applications.
**Key Features**
* These agents can converse, analyse, and act on the data. For instance, an agent might monitor sales velocity and auto-generate reorders or interact with a merchant in natural language to explain an inventory discrepancy.
* By combining RKG and vector retrieval (the “GraphRAG” approach), agents provide answers that are accurate and explainable in the retailer’s context. They can then trigger workflows or notifications as needed.
* This AI-native design means Ekyam is built from the ground up to support generative AI agents that perform end-to-end retail tasks.
### Chronicle Unified Schema
Chronicle is the powerhouse behind Ekyam, a sophisticated data engine designed to capture and make sense of all product and inventory activity across your entire retail ecosystem. Chronicle effortlessly normalizes events from diverse sources into a common format.
This transforms fragmented data into something truly valuable—it becomes searchable, comparable, and incredibly insightful. This comprehensive event timeline acts as a real-time, chronological record. It empowers your AI assistant to provide instant, up-to-date answers to critical questions like "What orders shipped yesterday?" or "What returns occurred last week?" More than just answers, Chronicle ensures your assistant maintains seamless conversation context, leading to more accurate and helpful interactions.
**Key Features**
* It combines structured relational storage—for facts like SKU-level stock counts, product attributes, and store inventory positions—with a vector database designed for unstructured data such as product descriptions, packaging notes, or supplier-uploaded spec sheets.
* All data normalized by the Retail Knowledge Graph (RKG) is written into Chronicle. For AI use cases, textual content (e.g., “color discrepancy in recent shipment” or “damage reason noted by receiver”) is broken into chunks and vectorized.
* Vector embeddings allow Ekyam agents to semantically search past records and retrieve similar inventory events—such as SKUs flagged for quality issues or suppliers with recurring item mislabels.
* In Retrieval-Augmented Generation (RAG) workflows, an agent may ask, “Have we had this issue before with SKU-123 in the past six months?” Chronicle then returns semantically relevant examples, which the AI model uses to generate a grounded recommendation. Acting as the AI’s long-term memory for retail product data, Chronicle enables intelligent, explainable decision-making that links structured inventory records with real-world context.
### GPT Style AI- Agents
In the Ekyam platform, GPT-style AI agents are intelligent, autonomous entities that use LLM capabilities to understand complex retail data, reason over the Retail Knowledge Graph (RKG), generate human-readable insights, and act across integrated systems. They enhance automation, provide deeper insights, reduce manual effort, and improve response times. The agent has short-term context and long-term memory, as well as tool-using capability. Ekyam's AI- Agents understands the user's requirements, breaks it down into simple structure, creates and runs a query, analyse the data and respond to the user.
### Vector Database (Semantic Memory)
A vector (embedding) store holds representations of unstructured or semi-structured information for long-term “semantic memory.” This includes embedding past conversations with the assistant, customer or product documents, and even summarized contents of Chronicle events. At runtime, the agent uses semantic search over the vector DB to recall relevant information (for example, pulling up similar past questions or related context). This enables the assistant to remember details across conversations and find relevant facts by meaning not just keywords.
### Internal Tooling / APIs
A suite of internal microservices or API endpoints (called “tools” by the AI agent) for data access and mutation. Examples include GetInventoryBySKU, GetOrderStatus, SummarizeReturns, CreateRestockOrder, etc. Each tool encapsulates a backend operation (often implemented via Ekyam APIs or direct queries to the RKG/Chronicle) and exposes a defined input/output schema. The agent can invoke these tools programmatically (through the LLM’s function-calling ability or agent toolkit) to retrieve live data or trigger processes. For instance, calling GetInventoryBySKU("SKU123") might query the RKG or inventory database and return current on-hand quantities.
# Universal Connectors
Source: https://docs.ekyam.ai/introduction/universal-connector
### **• Connecting your Ecosystem**
Ekyam’s **Universal Connectors** are adapters that integrate major retail systems into the platform. There are connectors for ERP/finance systems (NetSuite, Oracle, Dynamics), warehouse management (Manhattan WMS, etc.), order management (Shopify, Fabric OMS, etc.), POS, marketplaces (Amazon, Walmart), PIM systems, BI tools, Authentication Protocols (Oauth 1.0, Basic Authentication, SSO, Two-Factor Authentication), Redirect Flows, Different Data sources (Databases, Structured Documents, Cloud-File Storage, and XML and JSON files).
Each Universal Connector is capable of efficiently handling data mapping and transmission. For example, **the NetSuite connector** pulls item definitions and on-hand inventory; ***the Shopify connector*** ingests new orders and customer data; ***the Manhattan OMS connector*** provides order-fulfillment status updates.
These Universal Connectors feed ***two primary stores***: the **RKG **(for master/reference data) and **Chronicle** (for transactional events). In many cases, connectors also allow “action” operations: for instance, the agent might trigger a connector function to create a purchase order in the ERP.
Ekyam can seamlessly connect with virtually any system within the retail ecosystem, thereby ensuring comprehensive data collection and distribution. Ekyam’s **Universal Connector** is built on a micro-services layer, which offers support to different protocols and formats, enabling robust integration.
The different connectors for ERP, POS systems, WMS, E-commerce, CRMs, OMS, PIMs and others are listed below:
* SAP (S/4HANA, ECC)
* Oracle (EBS, Fusion Cloud)
* Microsoft Dynamics 365
* NetSuite
* Infor
* Custom/Legacy ERPs
* Lightspeed
* Square
* Shopify POS
* NCR
* Custom/In-house POS
* Shopify
* Magento / Adobe Commerce
* Salesforce Commerce Cloud (Demandware)
* WooCommerce
* BigCommerce
* Custom E-commerce solutions
* Manhattan Associates
* Blue Yonder (JDA)
* Oracle WMS Cloud
* SAP EWM
* Custom/In-house WMS
* Salesforce
* Microsoft Dynamics 365
* Zendesk
* ServiceNow
* IBM Sterling OMS
* Magento OMS
* Custom OMS
* Akeneo
* Salsify
* UPS, FedEx, DHL APIs
* 3PL (Third-Party Logistics) systems
* QuickBooks
* Xero
* SAP FICO modules
* Mailchimp
* Braze
***
# Universal Reader & Writer
Source: https://docs.ekyam.ai/introduction/universal-reader-writer
### **• Ekyam Universal Reader**
Ekyam’s Universal Reader’s primary function is to map the data from all connected systems to Ekyam’s proprietary Data Standards. Ekyam Data Standards are a crucial element of the platform. They represent a canonical, unified data model for all key retail entities and processes. The standardized representation ensures that the data is understood consistently within the Ekyam ecosystem. In addition, Ekyam’s Universal Reader is responsible for ingesting raw data from various source systems (including parsing EDI/iDOCs) and transforming it into Ekyam Data Standards.
**How does the Universal Reader transform the data into Ekyam’s Proprietary Data Standards?**
* **Mapping:** The Universal Reader takes raw data (e.g., a JSON payload from Shopify, an XML file from an old ERP, or rows from a database) and transforms it into the Ekyam standard format.
* **Normalization:** Data is cleaned, validated, and structured according to the Ekyam Data Standards. For instance, "SKU," "ItemCode," "Product ID," and "UPC" from different systems might all be mapped to a single, standardized "ekyamProductID" field.
**Use Case: NetSuite Reader Mapping**
This use case illustrates how Ekyam's Universal Reader facilitates the seamless ingestion and standardization of data from NetSuite, transforming it into Ekyam's proprietary data standards for unified storage and processing.
**The Ekyam Universal Reader Mapping Process:**
The Ekyam platform is configured to target NetSuite's data. Ekyam utilizes:
* NetSuite's /inventoryItem endpoint to pull comprehensive **Product** data.
* NetSuite's /inboundshipment endpoint to pull detailed **Shipment** data
1. Verify button verifies if end points used are valid or or invalid.
2. If the verification fails, below is the message displayed
**→ Predefine Collections (Left Panel)**
The vertical menu on the left showcases all predefined sub-collections or components under the primary Product entity. These include:
1. Pricing
2. Variants
3. Media
4. Inventory
5. Attribute
6. Marketing etc.
Each of these sub-collections encapsulates a logical group of related fields, allowing users to navigate and configure mappings.
**→ Mapping Panel (Right-Panel)**
The right side section presents a field by field mapping interface:
**Left column (Source Fields)**: These are Ekyam’s standard fields, which form the canonical data schema within the Ekyam platform. These fields are consistent across all integrations, serving as a “Single Source of Truth”.
**Right Column (Destination Fields):** These dropdowns represent field from external/destination system (eg. SAPB1, Netsuite etc) that are mapped to the corresponding Ekyam standard fields. The values are either selected manually or auto-filed based on the configuration or previous mappings.
* When the "Products" tab is selected from the left panel, the screen dynamically displays the specific field mappings for product data. Here, Ekyam's standardized keys for the Products collection are clearly defined, such as:
* Product ID
* Product SKU
* Pricing Tiers
* And many others...
* Once the synchronization process commences, the data pulled from NetSuite's **/inventoryItem endpoint** (for Products) is immediately subjected to these AI-driven key mappings. The raw NetSuite product data is transformed into Ekyam's standardized product format and subsequently saved to MongoDB.
* Similarly, data pulled from NetSuite's **/inboundshipment endpoint **(for Shipments) undergoes the same intelligent mapping process.
**AI-Powered Data Mapping**
To define how fields in a source system correspond to fields in a target/destination system, or how they map to a central standard, Ekyam significantly accelerates and simplifies this process through **Artificial Intelligence (AI)**. This AI-assisted mapping is applied in Universal Reader - mapping to Ekyam standards.
* **AI-Recommended Mappings:** When connecting new systems or defining data flows, Ekyam's AI engine analyzes the data schemas of the source and destination systems (or the Ekyam Data Standards). Based on field names, data types, and patterns, the **AI** **recommends potential mappings.**
**Custom Field Mappings**
While Ekyam’s AI-driven Universal Reader is capable of translating the majority of the data to its proprietary Data standards, there are instances where specific fields from NetSuite may not match within Ekyam’s pre-defined standard.
In such scenarios, Ekyam provides the capability for **Custom Mappings**. These custom mappings are then saved in Ekyam's database, ensuring that the Universal Reader consistently applies these specific rules during subsequent syncs.
### **• Ekyam Universal Writer**
Just as the Universal Reader ingests and standardizes data, the **Universal Writer** is responsible for delivering data from Ekyam to the various connected destination systems in the specific format and structure they expect.
Ekyam’s Universal Writer takes standardized data from within Ekyam and transforms/formats it for delivery to destination systems or trading partners, including generating EDI/iDOC documents.
The Universal Writer can:
* **Transform Ekyam Standardized Data:** Take data held in the Ekyam Data Standards (e.g., from the Universal Ledger or as a result of a workflow) and map it to the unique schema and format required by the receiving system (e.g., a specific XML structure for an ERP, a JSON payload for a marketing automation tool, or a CSV file for a reporting system).
* **Transmit External Data:** In some scenarios, data might flow directly from one external connected system to another, with Ekyam orchestrating the transfer and ensuring the data is correctly formatted by the Universal Writer for the destination system.
This capability ensures that while Ekyam uses its internal standards for processing and as a source of truth, it communicates with each external system in its native language, ensuring seamless integration and interoperability.
# RKG Powers AI Agents
Source: https://docs.ekyam.ai/rkg/agents
This document will give a walkthrough of how the Ekyam's Retail Knowledge Graph (RKG) is a powerful tool for AI agents. It provides the agents with the structured context and relationships to understand complex queries and generate relevant responses.
**Use of RKG in Agent Reasoning**
A Retail Knowledge Graph basically serves as an **“Authoritative Source of Truth”** for AI agents to provide structured and validated data, which agents can rely on for generating accurate responses. Furthermore, it acts as the “Retrieval” component in a Retrieval-Augmented Generation (RAG) system for retail applications. By acting as a Retrieval component, RKG stores interconnected retail entities (products, customers, transactions) that agents can query to augment their knowledge before generating responses.
**How do Agents Query the Graph**
The most crucial step of an AI agent is to retrieve relevant information before generating a response. When it involves the use of RKG, then the retrieval process becomes highly structured as it leverages the data from the graph.
Primarily, the agents use Cypher queries to traverse the graph, which helps in retrieving specific facts like product relationships, customer purchase history and sales patterns before formulating answers.
**How RKG powers AI Agent Reasoning**
The RKG’s graph structure allows agents to follow relationships across multiple hops. Below is the step-by-step explanation of how an agent uses the RKG for multi-step reasoning:
**Customer → Purchase → Product → Category → Supplier**
This process is a traversal of the graph. The agent is not performing multiple look-ups rather it is navigating a pre-existing, pre-computed web of relationships. This relationship allows to:
* **Connect Disparate Data:** The RKG links products, customers and transactions, and eliminates the need for complex connect.
* **Increase Efficiency:** To traverse a relationship in a graph is more efficient for complex, interconnected queries than connecting joints in a relational database.
**Use case**
**An agent can answer "Which customers bought this shirt in the last quarter and what were the total sales?" By traversing: Product → Sales Transactions → Customer Profiles → Aggregating sales data across time periods, all in a single graph query.**
With this semantic, interconnected data model, RKG transforms an AI agent from a data retriever to an intelligent responder.
# RKG Schema
Source: https://docs.ekyam.ai/rkg/rkg-schema
### **• Purpose of the Schema**
Ekyam’s Retail Knowledge Graph is a structured representation or graph database that models retail domain entities like (Products, SKUs, categories, vendors, locations, customers, orders etc) and their relationships. The Knowledge Graph is built on Neo4j and encodes the Semantic Schema (Ontology) of the retail domain (for instance, “Product A is supplied by Vendor X” or “SKU123 is stocked at Warehouse Y.”)
This structured representation provides a **“System of Truth”** for facts about products, vendors and static reference data. The Knowledge Graph provides an understanding, contextual linkage and graph-based querying over retail data sourced from **MongoDB** using Ekyam Standards.
Natural Language Queries about product details or vendor relationships are grounded in Ekyam’s Retail Knowledge Graph, which can be queried via graph query APIs or natural-language-to-graph translations.
### **• Why a Graph-based Approach?**
A Graph Database is a great way of representing and querying relationships between connected data. Graph databases are chosen to model complex, interconnected relationships (e.g., a customer → order → item → product) that are cumbersome in relational models.
A graph-based approach uses a graph structure with attributes, relationships and objects to represent data. Nodes are objects, edges demonstrate the relationship between those nodes, and properties describe the attributes of the nodes and edges. This dynamic structure makes a graph database useful for connected data representation. It offers more flexibility regarding relationships and data types.
At its core, a graph database uses a **graph structure** to represent information:
* **Nodes** act as the individual "objects" or entities (e.g., a customer, an order, a product).
* **Edges** show the connections or "relationships" between these nodes (e.g., a customer "placed" an order, an order "contains" an item).
* **Properties** provide details or "attributes" for both nodes and edges (e.g., a customer's name, an order's date, an item's quantity).
*While MongoDB stores raw structured data, Neo4j captures semantic relationships, enabling rich queries and reasoning that go beyond flat tables.*
### **• Indispensable Schema Benefits**
RKG is a structured knowledge graph that has a capability to interconnect data from disparate sources like inventory, e-commerce websites, product insights etc. This data is organized and structured in a single data management system.Its modular, ontology-driven design, ensures that new node types and relationships can be integrated without disrupting the existing graph structure.
The schema is designed to evolve seamlessly with the expansion of retail-entities, easily incorporating new elements like returns, loyalty programs or marketing channels. In addition, if the schema is well-defined, it will ensure data consistency, support interoperability across systems, enable querying and reasoning, thereby providing a clear conceptual model for the graph. It acts as a contract between data ingestion and application layers to minimize ambiguity.
**Who will benefit from this Schema**
The schema is intended for developers (for implementation and integration), data scientists (for advanced analytics and ML tasks), and business analysts (for querying business metrics and relationships).
### **• Core Principles**
Our Ekyam Retail Knowledge Graph is meticulously designed around several core principles that ensure its robustness, adaptability, and utility across a wide range of applications and future needs. These principles guide every decision in its architecture and implementation:
* **Flexibility and Extensibility:** Ekyam’s Knowledge Graph is capable of accommodating new data sources, and allows seamless incorporation of new domains, entities and relationships.
* **Scalability:** Ekyam’s Retail Knowledge Graph architecture handles vertical scalability (Increasing resources on a single machine) and horizontal scalability (Distributing data and processing across multiple machines).
* **Data Integrity:** An important principle is to ensure the accuracy, consistency and reliability of the data within the Knowledge Graph. This involves implementing validation rules, enforcing constraints (uniqueness, data types) and employing mechanisms for data reconciliation and error handling.
* **Ease of Querying:** Ekyam’s Knowledge Graph is designed to be easily accessible for querying by users, data scientists and developers to business analysts. This includes providing intuitive query languages (e.g., GraphQL, SPARQL, or even natural language interfaces), clear documentation, and tools that simplify data exploration.
### **• Core Entity Definitions**
### Node and Relationship Types
A **Node** in Ekyam’ Retail Knowledge Graph, represents a distinct entity or object in the retail ecosystem such as a Product, Customer or Order. Each node contains key attributes that describe the entity.
A relationship type in Ekyam Retail Knowledge Graph defines a connection or interaction between two nodes. For instance, when a customer places an Order, or a Product HAS\_VARIANT Item. Relationships are directional and may also include properties (eg. Timestamps, quantities).
**Nodes**\
Use PascalCase (Product, Customer, Shipment)
**Relationships**\
Use UPPER\_SNAKE\_CASE (PLACED, HAS\_VARIANT, SOLD\_ON)
***Relationship names describe actions or associations, while node names reflect entity types.***
**Key Entities**
* **Product:** It is a generic item for sale that represents a style or a SKU family.
**Key Attributes**
Ekyam captures a comprehensive set of key attributes for each product to ensure accurate representation. These include essential identifiers like:
| **Fields** | **Identifiers** |
| :--------------------------- | :----------------------------------------------------------------------------------- |
| **SKU (Stock Keeping Unit)** | The business-defined Stock Keeping Unit for the variant. |
| **Name** | Name of the product that will help the user to easily find the product |
| **Description** | Detailed description of the category for informational purposes. |
| **Short\_Description** | Concise summary of the product |
| **Status** | Status of the return process with options: pending\_approval, processing, completed. |
| **Brand Name** | Indicates the Name of the company or trademark associated |
| **List\_Price** | Object containing the amount and currency of the product's list price. |
| **Aggregate\_Stock** | Field representing the total, sellable, and reserved stock quantities for a product. |
| **Physical Attributes** | Field containing weight, dimensions, and units for physical attributes of a product. |
| **Tags** | Field storing descriptive tags associated with a product |
| **Compliances** | Field holding compliance information such as warranty details for a product. |
| **List\_price\_Currency** | Pricing information including amount and currency. |
| **Categories** | Array of categories the product belongs to with nested category details. |
| **Partners** | Array of partners associated with the product with nested partner details. |
| **Created\_at** | Date and time when the redemption record was created. |
| **Updated\_at** | Timestamp indicating the last update time of the redemption record. |
**Product - Item Relationships**
Ekyam defines a clear relationship between a generic Product and its specific Items. This allows the platform to model a product line where a single product concept can have multiple variations.
1. **(:Product) \[:HAS\_VARIANT]-> (:Item) →** A product can HAS\_VARIANT of one or more items. This means that a broader category (Men’s Tshirts) can have specific items associated with it.
2. **(:Item) \[:IS\_VARIANT\_OF]> (:Product):** An Item IS\_VARIANT\_OF of a single product.
### Inventory Snapshot
This section captures the inventory level of a product or variant at a specific timestamp. The Key attributes include:\
***SKU(Unique), Variant\_id, Product\_id, Name\_at\_sale, Price\_per\_unit, Quantity, Line\_item\_total, Color, Physical Attributes(Weight, Weight\_unit, Length, Width, Height, Dimension\_unit)***
### Product-Item Relationships
This outlines how different entities in a retail context are connected, representing a robust way to model complex relationships, often used in graph databases for efficient querying and insights.
**→ Product and Item (Variant Relationships):**
**(:Product) \[:HAS\_VARIANT]> (:Item)**: This models the concept of product variants.
**(:Item) \[:IS\_VARIANT\_OF]> (:Product):** It allows for easy navigation from a specific Item back to its general Product category.
**→ Order and Item Relationships:**
**(:Order)\[:HAS\_ITEM quantity, price, line\_item\_total]> (:Item):** This represents the line items of an order.
**(:Item) \[:BELONGS\_TO quantity, price, line\_item\_total]>(:Order):** Allows for easy querying to find all orders that contain a particular Item.
→**Item and Source Relationships:**
**(:Item) \[:SOLD\_ON]> (:Source):** Tracks the sales channel or platform where a specific Item was transacted.
**(:Source) \[:HAS]> (:Item):** Enables querying to find all Items sold through a specific Source.
→ **Shipment and Item Relationships:**
**(:Shipment) \[:CONTAINS\_ITEM quantity]> (:Item):** Depicts the actual physical movement of goods. A single Order might result in multiple Shipments (e.g., if items are from different warehouses), so tracking Item quantity within a Shipment is critical.\
\
**(:Item) \[:INCLUDED\_IN quantity]> (:Shipment):** Allows for tracing which Shipments a particular Item was part of.
### PurchaseOrder (PO)
The 'purchase\_orders' collection represents the purchase orders placed by a Business Entity with a Partner. The key attributes include:
Po\_id, business\_entity\_id, partner\_id, location\_id, type, status, order\_date, expected\_delivery, total\_amount, items, is\_active, created\_at, updated\_at, po\_number, business\_entity\_name, partner\_name, tax\_amount, shipping\_amount, grand\_total, payment\_terms, shipping\_method, notes, created\_by, approved\_by, approved\_date.
### PurchaseOrder Relationships
These relationships illustrate how a PurchaseOrder connects with other critical entities in your business ecosystem, providing a clear map of your procurement process.
**→** **(:PurchaseOrder) \[REQUESTED\_FROM] ->(:Partner):** A PurchaseOrder node is linked to a Partner node (representing a vendor or supplier) via the REQUESTED\_FROM relationship.
This clearly identifies which supplier the goods or services on the purchase order are being requested from.
→ **(:PurchaseOrder) \[SUPPLIES] -> (:Item):** A PurchaseOrder node is linked to one or more Item nodes via the SUPPLIES relationship.
This specifies which particular items (products, goods, services) are included and expected to be supplied by this purchase order. This relationship is fundamental for tracking incoming inventory and matching orders to receive goods.
### **• Ontology Reference**
Ekyam’s Ontology Reference establishes a foundational structure and relationships for all retail data ensuring seamless integration across every system. It is the blueprint that makes the data intelligent and interoperable.
There are a few Standardized identifiers that allow disparate systems to understand and communicate about the same entities.
* **Stock Keeping Units(SKUs)**
* **How are SKUs represented and linked?**
SKUs are represented as a property in both Product and Item nodes. The HAS\_VARIANT and IS\_VARIANT\_OF relationships link products to their specific SKUs, enabling precise tracking and variant-level analysis.
* **Locations**
Locations are encapsulated within the Customer node under the addresses list, storing structured fields like street, city, state, zip, and country. These allow geolocation-based insights or regional segmentation.
* **Other Key Ontologies**
1. UOMs are embedded within physical\_attributes in the Product node.
2. Currencies are defined in the Order node via the currency field.
3. Dates/Times are consistently ISO-8601 formatted and used across all temporal fields like created\_at, order\_created\_date, and payment\_date.
### **• Cross-Referencing and Data Linkage**
The true power of Ekyam’s ontology lies in cross-referencing and data linkage. This is how Ekyam can transform isolated pieces of information into a cohesive, intelligent graph.
**How different entities are connected through common identifiers and relationships to form a cohesive graph?**
Entities are connected using shared identifiers (e.g., product\_id, variant\_id, order\_id, customer\_id) through well-defined relationships. This creates a cohesive graph where each node is contextually linked e.g., items sold in orders, shipped in shipments, or placed by customers ensuring complete traceability and semantic richness.
### **• Field-Level Details**
Ekyam’s Retail Knowledge graph uses common data types:
1. **String**: IDs, names, status, emails, etc.
2. **Float**: Prices, totals, weights, dimensions, etc.
3. **Integer**: Quantities.
4. **Boolean**: Flags like default in addresses.
5. **Date/Time**: All timestamps are in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ).
6. **Enum**: For fields like status (active, CANCELLED, etc.).
**Optional/Required Fields**
***This clearly indicates which fields are mandatory and which are optional for each entity.***
**Required**: Unique IDs (e.g., product\_id, variant\_id, customer\_id, order\_id), key attributes (sku, status, name).
**Optional**: Descriptions, media, tags, compliances, wishlist, addresses, etc., which enhance context but are not mandatory for graph structure.
**Units**
This section establishes that all numerical fields ensure clear interpretation and consistency in data, whereas the Default units are pre-configured to streamline data integration.
The defaults are not hardcoded but assumed as:
1. status = "active" if not provided (by convention).
2. created\_at = current timestamp (if system-injected).
Otherwise, fields must be explicitly present to ensure completeness.
### **• Data Integrity Rules**
The Ekyam platform incorporates robust Constraints and Validations to ensure the highest level of accuracy. Our validation framework enforces:
→ All unique ID fields (\*\_id, order\_number, etc.)
→ Relationships require valid cross-referenced IDs (e.g., variant\_id in both Item and Order).
→ Enumerated fields are validated against allowed sets (like status, method).
→ Numeric fields (e.g., quantity, price) must be non-negative.
### **• Querying the EKG (via API or UI)**
The Ekyam Retail Knowledge Graph is queried via API endpoints using natural language. User queries are passed to a Gemini model along with the graph ontologies, and the model generates Cypher queries to be executed on Neo4j.
**API endpoints**
**Authentication and Authorization**
API access is currently open or controlled via environment-level variables, with potential for token-based or basic authentication in production.
**Query Language/Syntax**
The system uses natural language queries, which are translated into Cypher by the LLM.
Example:
User: "Show all orders placed by customer John."
**Cypher Generated:**
MATCH (c:Customer first\_name= 'John')\[:PLACED]>(o:Order) RETURN o
### **• Common Query patterns**
Ekyam’s graph-based data model empowers the user to query the retail information with greater flexibility. The user can perform queries that traverse complex relationships.
Here are some common, yet powerful, query patterns you can utilize:
* **Retrieving a Specific Node by ID:** It quickly accesses a particular entity when you know its unique identifier, thereby providing instant access to granular data for deeper analysis.
| MATCH (p:Product product\_id= '03946f2e-21c8-4476-bdf5-5b889cf702c5') RETURN p |
| :----------------------------------------------------------------------------- |
* **Traversing Relationships:** It helps in exploring connections between different entities. This enables understanding the flow of goods and facilitates complex relationship-based analysis.
| MATCH (v:Source)-\[:HAS]->(i:Item)-\[:BELONGS\_TO]->(p:Product) RETURN p |
| :----------------------------------------------------------------------- |
* **Filtering and Aggregation:** This refines the data sets and summarizes information based on the specific criteria and calculations, thereby helping in performance analysis, decision-making capability and summarizing large amounts of data into an explanatory insight.
| MATCH (c:Customer)-\[:PLACED]->(o:Order) WHERE o.status = 'COMPLETED' RETURN c.customer\_id, COUNT(o) AS total\_orders |
| :----------------------------------------------------------------------------------------------------------------------- |
### **•** **Is there a RKG User Interface?**
Currently, there is no dedicated UI; the system is API-driven. However, data can be visually explored using Neo4j’s built-in Browser or Bloom tools.
While no UI exists, search and filter capabilities are achieved through natural language prompts and LLM-generated Cypher that handles filtering conditions.
### **• Best Practices for Querying**
The user can follow below mentioned best practices for querying:
1. Use clear and concise prompts to guide LLM in generating precise Cypher.
2. Include unique IDs or names when referring to specific entities to avoid ambiguity.
3. Limit result sets with LIMIT for performance and clarity.
4. Avoid over-fetching deeply nested relationships unless required.
# Access Control (IAM)
Source: https://docs.ekyam.ai/security-compliance/access-control
Ekyam adheres to a **Zero-Standing-Access principle**, a policy which prohibits persistent or “standing” access to production environments.
We also have a **Just-in-Time (JIT) “Break-Glass” Protocol**, where different access is provided to different users like:
→ **Request:** An audited ticket should be submitted by the technical team specifying the purpose, duration, and resources.
→ **Approval (read-only):** Temporary Read-only access required from EM’s or DevOps for troubleshooting.
→ **Write/Emergency Access:** Requires mandatory approval from our Director of Engineering for any sensitive write access for emergency fixes.
→ **Execution:** Sessions are logged, supervised and are adhered to the four-eye-principle (second approver). The access gets automatically expired after the approved duration.
# Auditing, Monitoring, & Incident Response
Source: https://docs.ekyam.ai/security-compliance/auditing-monitoring
Our security policy is built on a prepared approach to handle the incidents proactively. At Ekyam, we handle our events by:
→ **Logging and Auditing:** All application logs, infrastructure events, access records are centralized in GCP Cloud Audit logs.
→ **Monitoring:** Proactive monitoring of application health, security events, and system performance. We have automated alerts to notify the incident team of potential threats or anomalies.
→ **Incident Response:** We maintain a formal Incident Response Plan (IRP) that contains procedures to take swift and organized action for any potential security threat. We notify our clients with a prompt response in accordance with our regulatory requirements.
### **Importance of Audit Logs**
* **Audit Logs & Data Retention**\
Ekyam implements the critical security controls to manage and protect the data as well as system activities. These controls identify and classify the data according to their sensitivity. Once the data has been classified as sensitive or confidential, the mandate standards are applied to such information.
### **Audit logs (Logging and Monitoring)**
We maintain the Audit logs in a chronological order to record the user activities that are relevant to the security. The events are securely recorded to keep a history of actions/activities within the system and applications.
Ekyam uses Centralized logging via GCP (Google Cloud Platform) Audit logs to manage the applications, infrastructure, and access events.
Our Audit logs provides a complete audit-trail of the logs and enables the incident team to monitor the environment, perform checks, determine threats and action immediately if they see an incident. In addition to performing all the activities, it is necessary for the incident team to comply with all the regulatory requirements.
### **Why are the Audit logs critical?**
We maintain the audit logs because of the following reasons:
* **Security Monitoring:** We monitor logs in real-time to detect unauthorized access or malicious activity. We monitor logs in real-time to detect unauthorized access or malicious activity.
* **Incident Response:** This activity provides support in monitoring the incidents and quickly responding when a security event occurs.
* **Forensic Analysis:** It is important to analyze the past events that might have caused security breaches. In other words, a process to collect the evidence of how the system was attacked by any malware.
### **Best Practices for Audit logs**
Ekyam ensures that the audit logs are utilized in an effective manner in order to support the compliance requirements and record incident responses.
* **Immutability:** The logs configured are tamper-proof and write-only to ensure they adhere to the compliance standards. When an event is recorded via **GCP Cloud Audit Logs**, it cannot be deleted or altered, ensuring that the audit trail can be used for forensic analysis and compliance verification.
* **Centralized Storage:** Ekyam uses the GCP Cloud Audit Logs to store infrastructure, application and access events in a secure and centralized location. This ensures that the data is secure even if a source system is compromised. This practice will also ensure that analysis and auditing can be done easily.
* **Real-Time Monitoring:** Ekyam ensures continuous monitoring of logs to check for any authentication attempts or unauthorized configuration changes. If there is any observation seen by the incident team, then triggers or alerts are sent to notify the security teams. This practice reduces the time taken to detect a threat.
* **Regular Reviews:** Scheduled reviews of the log data are conducted by the Ekyam security team. This analysis helps to identify security threats, and unauthorized access.
# Compliance Standards
Source: https://docs.ekyam.ai/security-compliance/compliance-standards
Ekyam follows a robust and a strict approach of adhering to the compliance program for maintaining the privacy, security and integrity of all customer data. It is an operational imperative for us to protect the confidentiality of customer’s data, limit unauthorized access and follow all protective measures to secure data. We have achieved compliance for:
### **SOC 2**
Ekyam mandates this auditing standard for protecting client’s data and keeping it secure. This is a non-negotiable and a vital requirement for all SOC2 reports. We adhere to this standard by keeping the client’s data protected against unauthorized disclosure, unauthorized access and maintaining the privacy of the data. We are committed to follow **Five Trust Criteria Principles**(TSCs).
* **Security:** This is the most fundamental Trust Criteria mandate for Ekyam as we secure the client’s data by restricting unauthorized access, monitor damage and prevent unauthorized disclosure of data. Ekyam’ security layer has four essential layers: **Access control**, **Incident response, Continuous Monitoring** and **Data Protection**.
* **Availability:** Ekyam engineers this criteria in a way that the system must be accessible and operational to authorized users. It also means that the system must be capable of recovering from any failure with a minimal impact. We have defined health checks for our services to help the system recover from failure mechanisms.
* **Confidentiality:** Ekyam adheres to the criteria of confidentiality by protecting the sensitive and confidential information from unauthorized users. We always store the confidential data in a secure environment and use designated encrypted keys for highly confidential data. The security team also runs automated checks to ensure that no confidential data has been shifted to a less-secure location.
* **Processing Integrity:** Ekyam’s integrity of processing criterion dictates that the data needs to be processed securely and accurately. We follow an approach where there is a mandatory input and output validation, which requires the security team to do a field verification, and check fields. Any data that does not fit into the criteria or fails the validation, the team records a detailed error log to manage the issue.
* **Privacy:** According to Ekyam’s defined and strict compliance standards, this criteria governs the regulated handling of Personal Identifiable Information (PII). We use the client’s data by adhering to the strict regulations of the standards and also by taking the consent of the user. Ekyam practices data minimization, therefore the necessary PII data is exposed for use.
### ISO 270001: 2022
Ekyam complies with the Global GOLD Standard for ensuring security of information supporting assets. We adhere to the **ISO 270001 Security Standard** for governing the organizational implementation of policies, procedures, and controls. In addition to this, Ekyam ensures that it supports companies in managing their information securely and maintaining confidentiality. Ekyam has successfully implemented the standard by moving through the PLAN, DO, CHECK, and ACT (PDCA) process.
### GDPR
Ekyam mandates the European Union (EU) regulation to protect the personal data and privacy of EU citizens. We ensure that all stages of the GDPR, including an initial assessment, gap analysis, implementation of changes, internal audits and external audits by third parties are managed efficiently. Ekyam assures that it keeps its stakeholders' information confidential, which strengthens its organizational credibility. By being GDPR-compliant, Ekyam ensures that it complies with the EU privacy laws; minimizes exposure to data breaches and non-compliance penalties; and strengthens relationships with clients through data protection.
### Ekyam’s Commitment to Data Rights and Protection
We follow strict adherence to the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA). Our mandate commitment to these standards defines our trusted relationship with our customers. We are built on \*\*Data Subject Rights \*\*that protects the user’s privacy as well as grants the users full control over their Personal Identifiable Identification (PII).
Ekyam’s formalizes its commitment to the standards and protections rights through \*\*Data Processing Agreements (\*\*DPAs). The agreement mentions the protection and security that we guarantee to our customers to manage their data in compliance with GDPR and CCPA standards.
*Our strict adherence to the Data Subject Rights along with the Data Processing Agreements makes Ekyam reliable, compliant and trustworthy to use the customer’s data for processing.*
# Data Retention Policy
Source: https://docs.ekyam.ai/security-compliance/data-retention-policy
The Data Retention Policy at Ekyam checks on how long the data, including the audit logs are stored. We have a strict data retention policy and deletion policies to avoid storing unnecessary data storage. Our policy complies with GDPR policies by securing and timely deleting the data that is no longer required for business requirements.
### **Why is Data Retention Policy critical?**
Ekyam ensures that a Data retention policy is critical for security and compliance as it directly implements the principle of data minimization. We ensure that different retention periods are defined for different data types so that the compliance requirements can be managed efficiently.
# Data Security & Privacy
Source: https://docs.ekyam.ai/security-compliance/data-security-privacy
Ekyam prioritizes Data security and privacy by enforcing strict encryption, data segregation and privacy policies.
### **Encryption**
Ekyam has robust encryption to protect data at every stage. This includes:
* **In-Transit:** All data is encrypted with TLS 1.2+ to prevent interception.
* **At Rest:** All Data, including databases, back-ups, and object storage is encrypted using AES-256 industry standards.
* Secret Management
Ekyam ensures that data is kept confidential, secure and private. This includes:
* Application secrets (API keys, decryption keys) are securely stored in the GCP Secret Manager with strict access controls and audit trails.
Under Data Security and Privacy, Ekyam also follows an Environment Data Policy:
**→ Production:** It contains live customer data and is subject to the highest level of protection.
**→ Staging:** Contains anonymized data for pre-release validation. No sensitive PII is used.
**→ Development:** Production data is never replicated in dev environments.
# Infrastructure and Network Security
Source: https://docs.ekyam.ai/security-compliance/infrastructure-network-security
Ekyam’s platform is built on Google Cloud Platform (GCP), and Microsoft Azure, both of which provide robust security-controls and mandate compliance with the industry standards. This ensures a scalable, and robust hosting for our services.
On **GCP,** Ekyam leverages a secure, scalable containerization through Google Kubernetes Engine (GKE) for deploying workloads. On **Azure,** Ekyam uses Azure Kubernetes Service (AKS) to ensure secure orchestration and workload isolation across environments.
Security is Ekyam’s priority and one of the **Core Components** of our security posture is robust **Network Isolation.**
**Network Segmentation and Isolation** uses Virtual Private Clouds in GCP and Azure for each environment (Development, Staging, and Production). Ekyam deploys **Firewall Policies** to restrict access control based on the **Principle of Least Privilege**, ensuring that only approved users can access the sensitive resources and prevent data from getting unauthorized access. Moreover, we have several systems deployed across multiple availability zones to ensure they can be **accessed securely** with authorized VPN access only. We have automated scaling and health checks to ensure that the system is able to recover from the failures.
Ekyam monitors, evaluates and strengthens its cloud security infrastructure through the best practices suggested by GCP and Azure to ensure that the customer’s data remains secure without impacting the operations.
# Introduction & Security Philosophy
Source: https://docs.ekyam.ai/security-compliance/introduction-security-philosophy
Ekyam’s commitment to security is a foundational principle of our technology platform. It is our fundamental commitment to protect the sensitive data entrusted to us by our customers. Our security framework is continuously validated by third-party audits. This includes achieving **SOC 2 Type I, Type II** compliance, **ISO 270001:2022** and **GDPR** standards which ensures that our security practices adhere to rigorous industry-related standards. We also build our models to adhere to the modern security models like:
* **Zero trust Architecture:** Every request received is authenticated, authorized and encrypted. In other words, Zero Trust implies that no access will be given to any user, or device from any location before it is verified and tested.
* **Principle of Least Privilege (PoLP):** Users and systems only have access to information that is important for them to function. The Least Privilege Access is an important part of Zero Trust Architecture and has a robust structure. It not only defines authentication but also strictly ensures authorization.
* **Defense in Depth:** It is one of the core principles of security where multiple layers of security from infrastructure to application code is applied so that even if one layer is compromised, the other layers remain intact.
# Secure Software Development Lifecycle (SDLC)
Source: https://docs.ekyam.ai/security-compliance/secure-software-development
At Ekyam, we follow a **Secure Software Development Lifecycle (SDLC)** to manage the code changes and also deploy the tested and approved modifications.
We ensure that all code and infrastructure changes are tracked in Git and should be submitted via Pull Requests (PRs). and undergo mandatory review. The PRs require mandatory approval from the **Team lead** or **Engineering Manager.**
Moreover, our **CI/CD Pipeline Security** is designed to ensure that the code deployment happens securely.
**→ Automation:** The approved code changes are managed by automated pipelines that build, test and deploy the code, thereby minimizing human error.
→ Our **Software Composition Analysis (SCA)** and **Integrated Static Application Security Testing (SAST)** tools scan for vulnerabilities in third party dependencies.
→ **Gated Deployments:** The production deployments are manually initiated yet a fully automated action, which requires a final approval from the **DevOps Lead**.
# Troubleshooting
Source: https://docs.ekyam.ai/support/troubleshooting
**• Common Errors & Troubleshooting**\
At Ekyam, we troubleshoot the errors of Authentication Failure, Data Mapping as mentioned below:
### Authentication Failure
* **Error Message Example:** 401 Unauthorized or Authentication Failed
* **Explanation:** A 401 Unauthorized error means that the system attempted to access an external service or API, but the credentials provided (e.g., API key, OAuth token) were either missing, invalid, or expired. The service denied access because it could not verify the identity or authorization of the request.
**Troubleshooting Steps**
1. **Check API Keys/Credentials**: Verify that the API keys, client IDs, client secrets, or other authentication credentials configured for the connector are correct and match what is expected by the external service.
2. **Refresh Tokens**: If using OAuth, the access token might have expired. Initiate the OAuth re-authentication process to obtain a new, valid token.
3. **Permissions**: Ensure the authenticated user or application has the necessary permissions to perform the requested operations on the external system.
4. **Network/Firewall**: In rare cases, firewall rules might be blocking the authentication handshake.
### Data Mapping Error
* **Error Message Example:** Missing Required Field: 'product\_name', Invalid Data Type for 'price', Schema Mismatch.
Explanation: A “Missing Required Field’ error occurs when the data does not match to the expected schema or requirements of a workflow step. This primarily implies that a mandatory field is absent or the data is an incorrect format.
### Troubleshooting Steps
→ **Review Mapping Configuration:** Navigate back to the data mapping configuration for the affected connector or workflow step.
→**Identify Missing Fields** : Compare the fields present in your source data with the required fields in the destination schema. Ensure all mandatory fields are correctly mapped.
→ **Check Data Types**: Verify that the data types of the mapped fields match the expected data types in the destination. For example, ensure a numerical field is not being sent as a string.
→ **Transformations**: If data needs to be transformed (e.g., concatenate fields, format dates), ensure the transformation logic is correctly applied before the data reaches the problematic step.
→ **Source Data Validation**: Examine the source data itself to ensure it contains the expected fields and formats.